Recording apparatus and method, playback apparatus and method, recording medium, and computer-readable medium for recording and playing back moving images

ABSTRACT

The present invention relates to a recording apparatus and a method, a playback apparatus and a method, a recording medium, and a program capable of quickly playing back a picture associated with a unit in which a moving image is encoded. An extraction section  51  extracts one picture from each GOP. A number-of-pixels conversion section  40  reduces the amount of information of the extracted picture by thinning out pixels. A static-image compression section  41  encodes the picture whose amount of information has been reduced by a JPEG technique. A microcomputer  31  associates the encoded picture with the GOP from which the picture has been extracted and controls recording of the picture associated with the GOP onto a disk  45  on which the moving image is to be recorded. The present invention can be applied to a recording apparatus for recording a moving image onto a data recording medium.

TECHNICAL FIELD

The present invention relates to recording apparatuses and methods,playback apparatuses and methods, recording media, and programs, and inparticular to a recording apparatus and a method, a playback apparatusand a method, a recording medium, and a program for recording movingimages on a data recording medium or playing back moving images recordedon a data recording medium.

BACKGROUND ART

It is a long time since consumer devices for digitally recording movingimages were commonly used. Nowadays, major data recording media forrecording such moving images are shifting from tapes to disks. Since(data of) moving images are randomly accessible by the use of datarecording media in the form of disks, images at positions far away onthe time axis for playback of the moving images can be quickly playedback. More specifically, so-called a playback start point can be locatedquickly and so-called nondestructively edited contents can now beproduced more easily by picking up and merging desired portions of amoving image.

To locate a playback start point or an edit point of a recorded movingimage, the moving image needs to be played back. This means that if themoving image is encoded, the moving image needs to be decoded before itcan be displayed.

However, when a moving image encoded as a program stream of the MPEG(Moving Pictures Experts Group) 2 technique is to be decoded, a portionto be decoded needs to be located and extracted from the data streambefore it can be decoded since the moving image is encoded in units ofGOP (Group of Pictures). Control required for this purpose iscomplicated and computationally time-consuming.

As described above, it is difficult to quickly perform the image at adesired point of an encoded moving image.

On the other hand, various recording techniques for recording encodedmoving images on a data recording medium are proposed.

FIG. 1 is a diagram depicting a recording technique for recording amoving image in a contiguous area of a data recording medium atpredetermined time intervals for playing back the moving image.

A stream unit 11-1 to a stream unit 11-6 are data produced by dividingthe moving image at predetermined time intervals for playing back themoving image. Each of the stream unit 11-1 to the stream unit 11-6 isrecorded in a contiguous area of the data recording medium.

When the moving image recorded by the recording technique shown in FIG.1 is to be read out from the data recording medium, the stream unit 11-1to the stream unit 11-6 are read out sequentially. When one of thestream unit 11-1 to the stream unit 11-6 is read out first and thenanother of the stream unit 11-1 to the stream unit 11-6 is to be readout, a seek time or a rotational latency is required between the twostream units.

FIG. 2 is a flowchart illustrating known processing for locating anddisplaying an edit point. In step S11, a management information filestoring management information indicating recording positions of themoving image on the data recording medium is read out. In step S12,stream data in the first frame is read out from the data recordingmedium in the form of a disk based on the read out managementinformation file.

In step S13, the read-out stream data is stored in a buffer. In stepS14, the encoded stream data is decompressed (decoded). In step S15,moving image data and audio data obtained by decompressing the streamdata are stored in the buffer downstream. In step S16, the moving imagedata and audio data are sequentially read out from the buffer downstreamto display a moving image based on the moving image data and outputaudio based on the audio data.

In step S17, it is determined whether a user has issued an instructionfor shifting to the subsequent point. If it is determined in step S17that the user has not issued an instruction for shifting to thesubsequent point, displaying the moving image and outputting the audioare continued and the flow returns to step S17 to repeat thedetermination processing.

If it is determined in step S17 that the user has issued an instructionfor shifting to the subsequent point, the flow proceeds to step S18,where stream data in the frame at the specified point is read out fromthe data recording medium in the form of a disk based on the read-outmanagement information file.

In step S19, the read-out stream data is stored in the buffer. In stepS20, the encoded stream data is decompressed (decoded). In step S21,moving image data and audio data obtained by decompressing the streamdata are stored in the buffer downstream. In step S22, the moving imagedata and audio data are sequentially read out from the buffer downstreamto display the moving image at the specified point based on the movingimage data and output the audio at the specified point based on theaudio data.

The procedure returns to step S17, where the processing of reading outstream data in the frame at the specified point from the data recordingmedium in the form of a disk according to an instruction from the userand decoding the stream data to display a moving image and output audiois repeated.

Also, there is another recording method for generating ClipMark composedof marks indicating distinctive images extracted from an input AV streamas management information for managing the AV stream, and furthermore,for generating PlayListMark composed of marks indicating any imagesspecified by a user from among playback sections corresponding toPlayList defining combinations of predetermined sections in the AVstream to record ClipMark and PlayListMark on a recording medium asseparate tables (e.g., refer to Patent Document 1).

[Patent Document 1] Japanese Unexamined Patent Application PublicationNo. 2002-158965

DISCLOSURE OF INVENTION Problems to be Solved by the Invention

As described above, when a start point or an edit point of an encodedmoving image is to be located, it is difficult to quickly display theencoded moving image since it needs to be decoded first before it can bedisplayed. As a result, locating a start point or searching for an editpoint takes a long time.

The present invention has been conceived in light of these circumstancesand is directed to quickly play back images associated with units ofimages, i.e., the units in which a moving image is encoded, to allow auser to view the content at a desired point in time for playback of themoving image.

Means for Solving the Problems

A recording apparatus according to the present invention includesextraction means for extracting an image from a unit in which a movingimage is encoded, the unit including a constant number of images;reduction means for reducing the amount of information of the extractedimage; encoding means for encoding the image whose amount of informationis reduced by a predetermined encoding scheme; association means forassociating the encoded image with the unit from which the image isextracted by the extraction means; and recording control means forcontrolling recording of the image associated with the unit onto a datarecording medium for recording the moving image.

The association means can be a track associated with a track of themoving image and can associate the encoded image with the unit byarranging the encoded image in a track in a predetermined file format.

The association means can associate the encoded image with the unit byassociating a range of time for playback of the unit of the moving imagewith the encoded image.

The recording control means can control recording of the moving imageonto the data recording medium such that the moving image in apredetermined time for playback is recorded in a first contiguous areaof the data recording medium and can control recording of the image ontothe data recording medium such that the image is recorded in a secondcontiguous area of the data recording medium when the amount of data ofthe encoded image exceeds a predetermined threshold if the recording ofthe moving image in the first area of the data recording medium isended.

The encoding means can encode the image by a compression and encodingscheme for a static image.

The encoding means can encode the image by a compression and encodingscheme for a moving image such that decoding is possible only with theimage.

The reduction means can reduce the amount of information of the image bythinning out pixels of the image.

The reduction means can reduce the amount of information of the image byremoving a high-frequency component of the image.

A recording method according to the present invention includes anextraction step of extracting an image from a unit in which a movingimage is encoded, the unit including a constant number of images; areduction step of reducing the amount of information of the extractedimage; an encoding step of encoding the image whose amount ofinformation is reduced by a predetermined encoding scheme; anassociation step of associating the encoded image with the unit fromwhich the image is extracted in the extraction step; and a recordingcontrol step of controlling recording of the image associated with theunit onto a data recording medium for recording the moving image.

A program on a first recording medium according to the present inventionincludes an extraction step of extracting an image from a unit in whicha moving image is encoded, the unit including a constant number ofimages; a reduction step of reducing the amount of information of theextracted image; an encoding step of encoding the image whose amount ofinformation is reduced by a predetermined encoding scheme; anassociation step of associating the encoded image with the unit fromwhich the image is extracted in the extraction step; and a recordingcontrol step of controlling recording of the image associated with theunit onto a data recording medium for recording the moving image.

A first program according to the present invention causes a computer toperform an extraction step of extracting an image from a unit in which amoving image is encoded, the unit including a constant number of images;a reduction step of reducing the amount of information of the extractedimage; an encoding step of encoding the image whose amount ofinformation is reduced by a predetermined encoding scheme; anassociation step of associating the encoded image with the unit fromwhich the image is extracted in the extraction step; and a recordingcontrol step of controlling recording of the image associated with theunit onto a data recording medium for recording the moving image.

A playback apparatus according to the present invention includes readingcontrol means for controlling reading an image from a data recordingmedium recording a moving image and the image, the image being extractedfrom a unit in which the moving image is encoded, the unit including aconstant number of images, the amount of information of the image beingreduced, the image being encoded by a predetermined encoding scheme, theimage being associated with each unit, and the reading being based on aninstruction from a user and a relationship with the unit of the movingimage; decoding means for decoding the read out image; and displaycontrol means for controlling display of the decoded image.

The reading control means can control reading the image from the datarecording medium so as to read only the image if the user directs afast-forward operation or a rewind operation.

The decoding means can decode the image encoded by a compression andencoding scheme for a static image.

The decoding means can decode the image encoded by a compression andencoding scheme for the moving image such that decoding is possible onlywith the image.

A playback method according to the present invention includes a readingcontrol step of controlling reading an image from a data recordingmedium recording a moving image and the image, the image being extractedfrom a unit in which the moving image is encoded, the unit including aconstant number of images, the amount of information of the image beingreduced, the image being encoded by a predetermined encoding scheme, theimage being associated with each unit, and the reading being based on aninstruction from a user and a relationship with the unit of the movingimage; a decoding step of decoding the read out image; and a displaycontrol step of controlling display of the decoded image.

A program on a second recording medium according to the presentinvention includes a reading control step of controlling reading animage from a data recording medium recording a moving image and theimage, the image being extracted from a unit in which the moving imageis encoded, the unit including a constant number of images, the amountof information of the image being reduced, the image being encoded by apredetermined encoding scheme, the image being associated with eachunit, and the reading being based on an instruction from a user and arelationship with the unit of the moving image; a decoding step ofdecoding the read out image; and a display control step of controllingdisplay of the decoded image.

A second program according to the present invention causes a computer toperform a reading control step of controlling reading an image from adata recording medium recording a moving image and the image, the imagebeing extracted from a unit in which the moving image is encoded, theunit including a constant number of images, the amount of information ofthe image being reduced, the image being encoded by a predeterminedencoding scheme, the image being associated with each unit, and thereading being based on an instruction from a user and a relationshipwith the unit of the moving image; a decoding step of decoding the readout image; and a display control step of controlling display of thedecoded image.

The recording apparatus may be an independent apparatus or may be ablock for performing recording in a recording and playback apparatus.The playback apparatus may be an independent apparatus or may be a blockfor performing playback in a recording and playback apparatus.

In the recording apparatus and method, first recording medium, and firstprogram according to the present invention, an image is extracted from aunit in which a moving image is encoded, the unit including a constantnumber of images; the amount of information of the extracted image isreduced; the image whose amount of information is reduced is encoded bya predetermined encoding scheme; the encoded image is associated withthe unit from which the image is extracted; and recording of the imageassociated with the unit onto a data recording medium for recording themoving image is controlled.

In the recording apparatus and method, second recording medium, andsecond program according to the present invention, reading an image froma data recording medium recording a moving image and the image iscontrolled, wherein the image is extracted from a unit in which themoving image is encoded, the unit including a constant number of images,the amount of information of the image is reduced, the image is encodedby a predetermined encoding scheme, the image is associated with eachunit, and the reading is based on an instruction from a user and arelationship with the unit of the moving image. Furthermore, theread-out image is decoded and display of the decoded image iscontrolled.

ADVANTAGES

As described above, according to the first present invention, imagesaccording to the moving image can be recorded on the data recordingmedium.

Furthermore, according to the first present invention, the imagesassociated with the unit can be quickly played back when the movingimage is to be played back. As a result, the user can view the contentat a desired point in time for playback of the moving image.

According to the second present invention, images according to themoving image can be played back.

Furthermore, according to the second present invention, imagesassociated with the unit can be quickly played back. As a result, theuser can view the content at a desired point in time for playback of themoving image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram depicting a known recording technique.

FIG. 2 is a flowchart illustrating the known process of locating anddisplaying an edition point.

FIG. 3 is a block diagram depicting a structure of one embodiment of arecording apparatus according to the present invention.

FIG. 4 is a diagram depicting extraction of one frame (picture) from oneGOP.

FIG. 5 is a diagram depicting one example of thumbnail data recorded ona disk.

FIG. 6 is a diagram depicting synchronization between thumbnail data andmultiplexed moving image data and audio data.

FIG. 7 is a diagram depicting an example of a file in PLF format.

FIG. 8 is a diagram depicting a file storing thumbnail data separatelyfrom a file in PLF format.

FIG. 9 is a diagram depicting an example of file in static image packageformat.

FIG. 10 is a diagram depicting an example of a file in track managementfile format.

FIG. 11 is a diagram depicting an example of a location-related datafile.

FIG. 12 is a diagram depicting locations.

FIG. 13 is a diagram depicting an example of a location-related datafile further referring to thumbnail data stored in an external file anda reference file storing thumbnail data.

FIG. 14 is a diagram depicting thumbnail data recording areas in whichthumbnail data is recorded.

FIG. 15 is a diagram depicting the process of recording thumbnail data81 adjacent to a stream unit on a disk.

FIG. 16 is a diagram depicting a thumbnail data recording area in whichthumbnail data is recorded.

FIG. 17 is a diagram depicting the process of recording onto a disk in acase where thumbnail data are recorded together at a position away froma stream unit.

FIG. 18 is a flowchart illustrating data conversion processing.

FIG. 19 is a flowchart illustrating data recording processing.

FIG. 20 is a block diagram depicting a structure of one embodiment of aplayback block in a recording and playback apparatus according to thepresent invention.

FIG. 21 is a flowchart illustrating the process of locating anddisplaying an edit point.

FIG. 22 is a diagram depicting an example of display of a thumbnail.

FIG. 23 is a diagram depicting reading out thumbnail data.

FIG. 24 is a diagram depicting reading out thumbnail data.

FIG. 25 is a diagram depicting a change in the amount of thumbnail datastored in a buffer memory during a fast-forward operation.

FIG. 26 is a diagram depicting a change in the amount of thumbnail datastored in a buffer memory during a rewind operation.

FIG. 27 is a diagram depicting reading out thumbnail data during arewind operation.

FIG. 28 is a diagram depicting in detail a change in the amount ofthumbnail data stored in a buffer memory during a rewind operation.

FIG. 29 is a flowchart illustrating the process of reading out thumbnaildata.

FIG. 30 is a diagram depicting storing thumbnail data in a buffermemory.

FIG. 31 is a diagram depicting storing thumbnail data in a buffermemory.

FIG. 32 is a diagram depicting storing thumbnail data in a buffermemory.

FIG. 33 is a diagram depicting storing thumbnail data in a buffermemory.

FIG. 34 is a diagram depicting storing thumbnail data in a buffermemory.

FIG. 35 is a block diagram depicting a structure of one embodiment of arecording and playback apparatus, according to the present invention,for reading out an MPEG2 program stream from a disk on which the MPEG2program stream is recorded and generating thumbnail data correspondingto the MPEG2 program stream to record it on the disk.

FIG. 36 is a block diagram depicting the structure of an I-pictureselection and decoding section.

FIG. 37 is a block diagram depicting the structure of a resolutionconversion section.

FIG. 38 is a block diagram depicting the structure of a JPEG encodingsection.

FIG. 39 is a flowchart illustrating the process of generating thumbnaildata.

FIG. 40 is a block diagram depicting a structure of one embodiment of arecording and playback apparatus, according to the present invention,for reading out an MPEG2 program stream from a disk on which the MPEG2program stream is recorded and generating thumbnails, as a stream of Ipictures, corresponding to the MPEG2 program stream to record it on thedisk.

FIG. 41 is a block diagram depicting the structure of an I-pictureencoding section.

FIG. 42 is a flowchart illustrating the processing of controlling theamount of code.

FIG. 43 is a block diagram depicting the structure of a VBV model.

FIG. 44 is a diagram depicting the motion of a VBV model in a case whereno restriction is applied to the amount of thumbnail data.

FIG. 45 is a diagram depicting the motion of a VBV model in a case wherethe amount of thumbnail data is restricted.

FIG. 46 is a block diagram depicting another structure of one embodimentof a recording and playback apparatus, according to the presentinvention, for reading out an MPEG2 program stream from a disk on whichthe MPEG2 program stream is recorded and generating thumbnails, as astream of I pictures, corresponding to the MPEG2 program stream torecord it on the disk.

FIG. 47 is a block diagram depicting the structure of an I-pictureselection and decoding section.

FIG. 48 is a block diagram depicting the structure of afrequency-characteristic conversion section.

FIG. 49 is a diagram depicting DCT coefficients.

FIG. 50 is a diagram depicting a transfer function H(n) of a horizontalfilter and a transfer function V(m) of a vertical filter.

FIG. 51 is a block diagram depicting the structure of an I-pictureencoding section.

FIG. 52 is a flowchart illustrating another process of generatingthumbnail data.

FIG. 53 is a flowchart illustrating another process of controlling theamount of code.

FIG. 54 is a block diagram depicting another structure of one embodimentof a playback block in a recording and playback apparatus, according tothe present invention, for playing back and displaying thumbnails basedon thumbnail data compressed and encoded in accordance with the JPEGtechnique.

FIG. 55 is a block diagram depicting the structure of a JPEG decodingsection.

FIG. 56 is a block diagram depicting another structure of one embodimentof a playback block in a recording and playback apparatus, according tothe present invention, for playing back and displaying thumbnails basedon thumbnail data compressed and encoded as a stream of I pictures.

FIG. 57 is a block diagram depicting the structure of an I-picturedecoding section.

FIG. 58 is a block diagram depicting still another structure of oneembodiment of a playback block in a recording and playback apparatus,according to the present invention, for playing back and displayingthumbnails based on thumbnail data compressed and encoded as a stream ofI pictures.

REFERENCE NUMERALS

31 microcomputer, 35 buffer memory, 38 moving-image compression section,40 number-of-pixels conversion section, 41 static-image compressionsection, 42 audio compression section, 43 buffer memory, 44 drive, 45disk, 48 disk, 51 extraction section, 81 thumbnail data, 101 file in PLFformat, 111 file, 121 file in static image package format, 131 trackmanagement file, 141 location-related data file, 151 location-relateddata file, 162 thumbnail data recording area, 203 moving-imagedecompression section, 204 static-image decompression section, 205 audiodecompression section, 206 image output interface, 302 I-pictureselection and decoding section, 303 resolution conversion section, 304JPEG encoding section, 305 file-format conversion section, 322 I-picturedetermination section, 323 selector, 324 variable-length code decoder,325 dequantization section, 326 inverse-DCT processing section, 341low-pass filter, 342 pixel-thinning section, 381 I-picture encodingsection, 402 visual-parameter detection section, 403 DCT processingsection, 404 control section, 405 quantization section, 406variable-length encoding section, 407 buffer, 451 I-picture selectionand decoding section, 452 frequency-characteristic conversion section,453 I-picture encoding section, 472 I-picture determination section, 473selector, 474 variable-length code decoder, 475 dequantization section,491 horizontal filter, 492 vertical filter, 501 control section, 502quantization section, 503 variable-length encoding section, 504 buffer,521 file-format conversion section, 522 JPEG decoding section, 561file-format conversion section, 562 I-picture decoding section, 591pixel-thinning section

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 3 is a block diagram depicting a structure of one embodiment of arecording apparatus according to the present invention. The recordingapparatus shown in FIG. 3 is constructed so as to include amicrocomputer 31 to a mode dial 46.

The microcomputer 31 is so-called a built-in microcomputer including,for example, a ROM (Read Only Memory), a RAM (Random Access Memory), aserial interface, or a parallel interface. The microcomputer 31 executesa predetermined control program to comprehensively control the recordingapparatus. The microcomputer 31 executes the predetermined controlprogram to issue an operating command to each section of the recordingapparatus based on a signal from a recording start/stop button 32 inaccordance with an operation of a user. The microcomputer 31 executesthe predetermined control program to adjust the file format of datastored in a buffer 43.

An image pickup section 33 includes an optical system, such as a lensand an aperture, and an imaging element, such as a CCD (Charge CoupledDevice) or a CMOS (Complementary Metal-Oxide Semiconductor) sensor andcaptures an image of a subject as a moving image to supply the imagesignal of the moving image thus obtained to a moving-image inputinterface 34. The moving-image input interface 34 is an interfacebetween the image pickup section 33 and a buffer memory 35. It convertsan image signal supplied from the image pickup section 33 into movingimage data in a predetermined format, such as applying analog-to-digitalor serial-to-parallel conversion to the image signal, and supplies theimage data to the buffer memory 35.

An audio conversion section 36 includes, for example, a microphone. Itacquires sound from the subject or sound surrounding the subject,supplies an audio signal corresponding to the acquired sound to an audioinput interface 37. The audio signal output from the audio conversionsection 36 is synchronized with an image signal output from the imagepickup section 33. The audio input interface 37 is an interface betweenthe audio conversion section 36 and the buffer memory 35. It converts anaudio signal supplied from the audio conversion section 36 into audiodata in a predetermined format, such as applying analog-to-digital orserial-to-parallel conversion to the audio signal, and supplies theaudio data to the buffer memory 35.

The buffer memory 35 includes, for example, a semiconductor memory andtemporarily stores image data supplied from the moving-image inputinterface 34 and audio data supplied from the audio input interface 37.The buffer memory 35 supplies the stored image data to a moving-imagecompression section 38 and a number-of-pixels conversion section 40.Furthermore, the buffer memory 35 supplies the stored audio data to anaudio compression section 42.

Under the control of the microcomputer 31, the moving-image compressionsection 38 compresses and encodes the image data for a moving imagesupplied from the buffer memory 35 through a predetermined technique andsupplies the compressed and encoded image data to a multiplexer 39. Forexample, the moving-image compression section 38 compresses and encodesimage data of a moving image supplied from the buffer memory 35 throughthe MPEG2 technique and supplies the compressed and encoded image datato the multiplexer 39.

Under the control of the microcomputer 31, the number-of-pixelsconversion section 40 extracts a predetermined picture (frame or field)from the image data of a moving image and converts the number of pixelsof the extracted picture. For example, the number-of-pixels conversionsection 40 converts the number of pixels of the extracted picture bythinning out pixels from the picture.

The number-of-pixels conversion section 40 is provided with anextraction section 51. For example, as shown in FIG. 4, the extractionsection 51 extracts one frame (picture) from one GOP of the image datafor a moving image compressed and encoded by the moving-imagecompression section 38 in accordance with the MPEG2 technique.

More specifically, for example, if the moving-image compression section38 compresses and encodes a moving image with 30 frames per second inunits of GOP composed of continuous 15 frames, the extraction section 51extracts one frame from the 15 frames constituting each GOP.

For example, the number-of-pixels conversion section 40 converts thenumber of pixels of the frame extracted from each GOP by thinning outpixels of the frame.

The number-of-images conversion section 40 supplies the image data whosenumber of pixels has been converted to a static-image compressionsection 41. Under the control of the microcomputer 31, the static-imagecompression section 41 encodes the image data supplied from thenumber-of-pixels conversion section 40 by means of a compression andencoding scheme for compressing static images. For example, thestatic-image compression section 41 encodes the image data supplied fromthe number-of-pixels conversion section 40 in accordance with the JPEG(Joint Photographic Experts Group) technique. The static-imagecompression section 41 supplies the encoded image data as thumbnail datato the buffer memory 43.

Under the control of the microcomputer 31, the audio compression section42 compresses and encodes the audio data supplied from the buffer memory35 by means of a predetermined technique and supplies the compressed andencoded audio data to the multiplexer 39. The audio data output from theaudio compression section 42 is synchronized with the image data outputfrom the moving-image compression section 38. For example, the audiocompression section 42 compresses and encodes the audio data suppliedfrom the buffer memory 35 in accordance with the AC3 (Audio Code Number3 (Dolby Digital™) technique and supplies the compressed and encodedaudio data to the multiplexer 39.

The multiplexer 39 multiplexes the image data supplied from themoving-image compression section 38 and the audio data supplied from theaudio compression section 42 and supplies the multiplexed image data andaudio data to the buffer memory 43. For example, the multiplexer 39multiplexes image data and audio data as an MPEG2 system stream formatand supplies to the buffer memory 43 the data in MPEG2 system streamformat composed of the image data and audio data generated bymultiplexing.

The buffer memory 43 temporarily stores the multiplexed image data andaudio data supplied from the multiplexer 39, as well as the thumbnaildata supplied from the static-image compression section 41.

The microcomputer 31 adjusts the format of the thumbnail data stored inthe buffer memory 43 into a predetermined file format. The file formatof thumbnail data will be described later with reference to FIG. 7 toFIG. 13.

A drive 44 reads out the multiplexed image data and audio data, as wellas the thumbnail data converted into a predetermined file format, fromthe buffer memory 43 and records them on a disk 45, which is one exampleof a data recording medium. The disk 45 is, for example, a magneticdisk, an optical disk, or a magneto-optical disk.

FIG. 5 is a diagram depicting one example of thumbnail data recorded onthe disk 45. Each of thumbnail data 81-1 to thumbnail data 81-n is datafor displaying one thumbnail. On the disk 45, in a case where errorcorrection is carried out by the ECC (Error Correction Coding) in unitsof 12 kilobytes, each of the thumbnail data 81-1 to the thumbnail data81-n is compressed to a size equal to or smaller than kilobytes.

Here, the unit in which data is subjected to error correction by, forexample, the ECC (Error Correction Coding) is recorded in one cluster,which is a unit in which records of data are managed.

Each of the thumbnail data 81-1 to the thumbnail data 81-n is recordedon the disk 45 in one cluster as the unit in which error correction iscarried out by the ECC. In this case, if any of the thumbnail data 81-1to the thumbnail data 81-n is smaller than 12 kilobytes, a data stringis added to the data that is smaller than 12 kilobytes of the thumbnaildata 81-1 to the thumbnail data 81-n so that the size of the data isincreased to 12 kilobytes. Each of the thumbnail data 81-1 to thethumbnail data 81-n increased to 12 kilobytes as a result of a datastring being added is recorded on the disk 45 in one cluster as the unitin which error correction is carried out by the ECC.

In the example shown in FIG. 5, since the thumbnail data 81-1 is smallerthan 12 kilobytes, a data string is add so that the size of thethumbnail data 81-1 is increased to 12 kilobytes and then the 12kilobytes thumbnail data 81-1 is recorded in one cluster.

In the example shown in FIG. 5, since the thumbnail data 81-n is 12kilobytes, no data string is added and the thumbnail data 81-n isrecorded as-is in one cluster.

In this manner, since it is sufficient to read data from one clusterwhen one of the thumbnail data 81-1 to the thumbnail data 81-n is to beread out, the thumbnail data 81-1 to the thumbnail data 81-n can be readout more quickly from the disk 45.

Hereinafter, the thumbnail data 81-1 to the thumbnail data 81-n arereferred to just as the thumbnail data 81 if it is not necessary todifferentiate them from one another.

The mode dial 46 supplies a signal for specifying the operation mode ofthe recording apparatus to the microcomputer 31 according to anoperation of the user. By changing the operation mode, for example, thenumber of pixels in one frame (picture) of recorded image data ischanged or whether thumbnail data is recorded or not can be changed.

A drive 47 is installed in the recording apparatus as required. From adisk 48 on which the control program is recorded, the drive 47 reads outthe program and supplies it to the microcomputer 31. The microcomputer31 stores the program read out from the disk 48 on the built-inrewritable ROM or RAM and executes the program. The functions of thedrive 47 and the disk 48 can be achieved by the drive 44 and disk 45.

FIG. 6 is a diagram depicting synchronization between the multiplexedmoving image data and audio data and the thumbnail data. In FIG. 6, anMPEG track is composed of moving image data and audio data, and athumbnail track is composed of thumbnail data. In FIG. 6, one rectangleindicates one image. The term “track” means a sequence of images orsounds.

For example, if moving image data constituting data in MPEG2 systemstream format is encoded in units of GOP composed of 15 frames andthumbnail data is generated from one of the frames in each GOP, one GOPin an MPEG2 system stream corresponds to one item of thumbnail data. Inthis case, one item of thumbnail data corresponds to a period of 0.5second in playback of the moving image.

The file format of the thumbnail data 81 associated with one GOP willnow be described with reference to FIG. 7 to FIG. 13.

As a file format for the thumbnail data 81, the Quick Time™ file formatcan be used. Hereinafter, the Quick Time™ file format is referred to asthe QT file format.

In the QT file format, moving image data, audio data, static image data,or the like is individually blocked, and furthermore, managementinformation for managing such blocked moving image data, audio data,static image data, or the like is also individually arranged intoblocks. Such a block is a basic data unit and is called an atom. Blockedmoving image data, audio data, static image data, or the like is managedon a track-by-track basis, and the information is called a track atom.Furthermore, information for managing a plurality of tracks as one itemof moving image data is called a movie atom.

One movie data atom corresponds to one track.

FIG. 7 is a diagram depicting an example of a file in Playlist File(PLF) format, as one example of QT file format, for storing thethumbnail data 81. In the head of a file 101 in PLF format for storingthe thumbnail data 81, data (file type data in the figure) fordescribing the file type is arranged. The data for describing the filetype is followed by data (profile data in the figure) for describing thefile profile.

For example, the data for describing the file type can be in a format incompliance with the MP4 extension (ISO14496-14) of BaseMediaFileFormat(ISO14496-12) of the ISO (International Organization forStandardization). In addition, for example, a value indicating the PLFformat is set in the data for describing the file profile.

In the file 101 in PLF format, the data for describing the file profileis followed by a movie atom (data indicated by moov in the figure). Thetrack atom (data indicated by trak (vide) in the figure) of a videotrack arranged in the movie atom of FIG. 7 is management information ofthumbnail data 81. A track atom (data indicated by trak (MPEG2 programstream) in the figure) of an MPEG2 system stream (MPEG2 program stream)track arranged in the movie atom of FIG. 7 is management informationabout the MPEG2 system stream in which moving image data and audio dataare multiplexed.

In a media atom (data indicated by mdia in the figure) of the trackatom, management information for managing the compression scheme,storage location, display time, and the like of the corresponding moviedata atom is stored. In a media information atom (data indicated by minfin the figure) of the media atom, various types of informationassociated with a sample, which is the minimum management unit, isarranged. For example, in the MPEG2 system stream program stream) track,the sample is one frame, and in the video track of the thumbnail data81, the sample is one item of thumbnail data 81.

In a sample table atom (data indicated by stbl in the figure) of themedia information atom, various types of information associated withindividual samples is arranged. In a time sample atom (data indicated bystts in the figure) of the sample table atom, a relationship betweeneach sample and a time in playback is described. In a sample chunk atom(data indicated by stsc in the figure) of the sample table atom, arelationship between samples and a chunk composed of the samples isdescribed.

Here, the term “chunk” means a data unit, in a track, composed of acollection of a plurality of samples.

In addition, in a sample size atom (data indicated by stsz in thefigure) of the sample table atom, the amount of data in each sample isdescribed. In a chunk offset atom (data indicated by stco in the figure)of the sample table atom, information about the location of each chunkwith respect to the head of the file is described.

Furthermore, the thumbnail data 81-1 to the thumbnail data 81-n arestored in the file 101 in PLF format as a movie data atom (dataindicated by mdat in the figure). In the file 101 in PLF format, thethumbnail data 81-1 to the thumbnail data 81-n are sequentiallyarranged.

More specifically, in the time sample table atom of the track atom ofthe movie atom shown in FIG. 7, a time in playback of each of thethumbnail data 81-1 to the thumbnail data 81-n, which are the movie dataatom, is described.

Because of this, each of the thumbnail data 81-1 to the thumbnail data81-n can be played back in conjunction with one GOP of the MPEG2 systemstream, as shown in FIG. 6.

The number of files recorded on the disk 45 can be reduced by storingthe thumbnail data 81-1 to the thumbnail data 81-n in the file 101 inPLF format as described above.

FIG. 8 is a diagram depicting a file storing the thumbnail data 81separately from the file 101 in PLF format. A file 111 shown in FIG. 8is a file for storing the thumbnail data 81-1 to the thumbnail data 81-nthat is referred to from the file 101 in PLF format. In the file 111,the thumbnail data 81-1 to the thumbnail data 81-n are sequentiallyarranged.

In this case, in the media information atom (data indicated by minf inthe figure) of the track atom in the file 101 in PLF format, managementinformation for referring to the file 111, such as the storage location(path and file name) of the file 111, is stored.

By doing so, instead of storing the thumbnail data 81 as the movie dataatom in the file 101 in PLF format, the thumbnail data 81 can berecorded as the file 111 in a unique format externally referred to andplayed back based on the file 101 in PLF format. Also in this case,since a time in playback of each of the thumbnail data 81-1 to thethumbnail data 81-n stored in the file 111 is described in the file 101in PLF format, each of the thumbnail data 81-1 to the thumbnail data81-n can be played back in conjunction with one GOP of the MPEG2 systemstream, as shown in FIG. 6.

In addition, the thumbnail data 81 can also be stored in a file instatic image package format as a file referred to by the file 101 in PLFformat.

FIG. 9 is a diagram depicting an example of a file in static imagepackage format for storing the thumbnail data 81. A file 121 in staticimage package format, which is a file in static image package format,has the same data structure as that of the file 101 in PLF format. InFIG. 9, the same data as that shown in FIG. 7 is denoted by the samename, and thus a description thereof will be omitted.

A value indicating the static image package format is set to the datafor describing the file profile in the file in static image packageformat.

In the file 121 in static image package format, a-track atom (dataindicated by trak (vide) in the figure), which is management informationfor the thumbnail data 81, is stored. Since the file 121 in static imagepackage format is a file referred to by the file 101 in PLF format, thetrack atom of the MPEG2 system stream is not stored in the file 121 instatic image package format.

The track atom in the file 121 in static image package format isdescribed as in the track atom of the file 101 in PLF format.Furthermore, in the file 121 in static image package format, thethumbnail data 81-1 to the thumbnail data 81-n are stored as the moviedata atom (data indicated by mdat in the figure).

Furthermore, the thumbnail data 81-1 to the thumbnail data 81-n may bestored in a file in location-related data file format, which is a fileformat for storing a plurality of items of metadata or image dataindividually associated with each location, serving as a time range of amoving image, and a time in playback may be described by means of a filein track management file format for storing track-related managementinformation.

FIG. 10 is a diagram depicting an example of a file in track managementfile format in this case. FIG. 11 is a diagram depicting an example of afile in location-related data file format in this case.

Since a video track cannot be used in a track management file 131 shownin FIG. 10, the first track atom (data indicated by trak (time locationdata) in the figure) in the track management file 131 is managementinformation for a location-related data file 141 shown in FIG. 11, andthe subsequent track atom (data indicated by trak (MPEG2 program stream)in the figure) in the track management file 131 is managementinformation for the MPEG2 system stream in which moving image data andaudio data are multiplexed.

In the track atom for the location-related data file 141, a samplecorresponds to each item of data for a plurality of locations stored inthe location-related data file 141. In other words, information abouteach item of data for a plurality of locations in the location-relateddata file 141 is stored in the media information atom (data indicated byminf in the figure) of the track atom for the location-related data file141. The relationship between each item of data for a plurality oflocations in the location-related data file 141, corresponding to asample, and a time in playback is described in the time sample atom(data indicated by stts in the figure) of the sample table atom of themedia information atom in the track atom for the location-related datafile 141.

As shown in FIG. 11, data is stored at each location in thelocation-related data file 141.

In the current description, the term “location” means a time range in amoving image, as shown in FIG. 12, and a plurality of locations issequentially arranged in order without overlapping each other or havinggaps therebetween. In short, a range in time can be specified byspecifying locations in sequence.

When the track management file 131 and the location-related data file141 are to be used, one thumbnail (thumbnail data 81) is associated withone location.

For example, as shown in FIG. 12, a thumbnail 1 (e.g., the thumbnaildata 81-1) is associated with a first location 1, a thumbnail 2 (e.g.,the thumbnail data 81-2) is associated with a location 2 subsequent tothe location 1, a thumbnail 3 (e.g., the thumbnail data 81-3) isassociated with a location 3 subsequent to the location 2, andsimilarly, a thumbnail n (e.g., the thumbnail data 81-n) is associatedwith n-th location n.

As shown in FIG. 11, a data number indicating the order of data and unitmetadata (metadata unit) are arranged in the location-related data file141 as data for each location. In the unit metadata, the data amount ofunit metadata, language used for description, metadata encoding scheme,data-type identification number for identifying the type of themetadata, the thumbnail data 81 serving as the metadata, and data otherthan the thumbnail data 81 are arranged in sequence. Data other than thethumbnail data 81 arranged next to the thumbnail data 81 may or may notbe stored in the unit metadata.

Therefore, each of the thumbnail data 81-1 to the thumbnail data 81-ncan be played back in conjunction with a location by storing managementinformation for referring to the location-related data file 141, such asthe storage location (path and file name) of the location-related datafile 141, in the media atom (data indicated by mdia in FIG. 10) of thetrack atom of the track management file 131 in track management fileformat and by arranging information about each item of data for thelocation in the location-related data file 141 (e.g., the number of thedata and information indicating the relationship between the number ofthe data and a time in playback) in the sample table atom (dataindicated by stbl in FIG. 10).

If the location is the time at which each GOP of the MPEG2 system streamis played back, each of the thumbnail data 81-1 to the thumbnail data81-n can be played back in conjunction with one GOP of the MPEG2 systemstream, as shown in FIG. 6.

Alternatively, the thumbnail data 81 stored in an external file mayfurther be referred to from a file in location-related data file format,rather than storing the thumbnail data 81 in the file inlocation-related data file format.

FIG. 13 is a diagram depicting an example of a location-related datafile 151 for further referring to the thumbnail data 81 stored in anexternal file without storing the thumbnail data 81 therein and areference file 111 storing the thumbnail data 81.

In the location-related data file 151, the data number and unit metadata(metadata unit) are arranged as data for each location. In the unitmetadata, the data amount of unit metadata, language used fordescription, metadata encoding scheme, data-type identification numberfor identifying the type of the metadata, and the metadata are arrangedin sequence. The metadata of the location-related data file 151 iscomposed of the file name (including the path) of the file 111, offsetof each item of the thumbnail data 81 in the file 111, and the data sizeof each item of the thumbnail data 81 in the file 111.

The offset in the metadata indicates the amount of data from thebeginning of the file 111 to the beginning of the thumbnail data 81referred to from the metadata. The data size in the metadata indicatesthe data amount of the thumbnail data 81 referred to from the metadata.

The layout on the disk 45 of the MPEG2 system stream and the thumbnaildata 81 recorded on the disk 45 will now be described.

The MPEG2 system stream is recorded in a contiguous area on the disk 45at predetermined intervals for playing back the moving image of theMPEG2 system stream.

A stream unit 161-1 to a stream unit 161-6 shown in FIG. 14 indicateunits in which the MPEG2 system stream is recorded, i.e., predeterminedtime intervals of 10 second to 20 second, for playing back the movingimage. The MPEG2 system stream is divided into one recording unit (e.g.,any of the stream unit 161-1 to stream unit 161-6) at predeterminedintervals for playing back the moving image and is recorded on the disk45. In other words, each of the stream unit 161-1 to the stream unit161-6 is data of the moving image generated by dividing the moving imageat predetermined intervals for playing back the moving image.

Hereinafter, the stream unit 161-1 to the stream unit are referred tojust as the stream unit 161 if it is not necessary to differentiate themfrom one another.

The stream unit 161 is recorded in one contiguous area on the disk 45.

As shown in FIG. 14, the thumbnail data 81 is recorded in a thumbnaildata recording area 162-1 and a thumbnail data recording area 162-2,which are areas contiguous to the respective stream units 161. Forexample, the thumbnail data recording area 162-1 and the thumbnail datarecording area 162-2 are provided adjacent to the front portions of therespective stream units 161 at the physical address of the disk 45.

By doing so, when the thumbnail data 81 is read out and then the streamunit 161 is read out, the stream unit 161 can be read out immediatelyafter the thumbnail data 81 has been read out, without requiring a seektime or disk rotational latency. This technique allows the number ofseek operations or the number of rotational delays in a case where thethumbnail data 81 is recorded to be the same as the number of seekoperations or the number of rotational delays in a case where thethumbnail data 81 is not recorded. Thus, this recording technique issuitable if the disk 45 is a disk having a relatively longer access time(seek or rotational latency), such as an optical disk.

Hereinafter, the thumbnail data recording area 162-1 and the thumbnaildata recording area 162-2 are referred to just as the thumbnail datarecording area 162 if it is not necessary to differentiate them fromeach other.

FIG. 15 is a diagram depicting the process of recording the thumbnaildata 81 adjacent to the stream unit 161 on the disk 45.

The buffer memory 43 is provided with a buffer for storing the MPEG2system stream and a buffer for storing the thumbnail data 81individually. For example, the two buffers in the buffer memory 43 maybe provided as separate hardware devices. Alternatively, the two buffersmay be logically provided by dividing an area into two at apredetermined address of the buffer memory 43 in the form of a singlehardware device.

The upper graph in FIG. 15 depicts a time-lapse change in the amount ofbuffered MPEG2 system stream data, and the lower graph in FIG. 15depicts a time-lapse change in the amount of buffered thumbnail data 81.The vertical axis in FIG. 15 denotes the amount of data, and thehorizontal axis in FIG. 15 denotes time.

When time t1 is reached a predetermined time after recording has beenstarted, the amount of buffered MPEG2 system stream data is equal to orlarger than the threshold for starting system stream recording.Therefore, recording of the buffered MPEG2 system stream onto the disk45 as the stream unit 161-1 is started. At time t2, since recording ofthe MPEG2 system stream has reached the end of the stream unit 161-1, aseek operation is performed or disk rotation is awaited during theperiod from time t2 to time t3. Thereafter, recording of the bufferedMPEG2 system stream in the subsequent stream unit 161-2 is started attime t3.

It is assumed that the amount of buffered thumbnail data 81 is equal toor larger than the threshold for starting thumbnail data recording attime t4, at which the MPEG2 system stream is being recorded in thestream unit 161-2. While recording the MPEG2 system stream on the disk45, the recording apparatus does not monitor the amount of thumbnaildata 81.

Since recording of the MPEG2 system stream has reached the end of thestream unit 161-2 at time t5, the recording apparatus determines at timet5 whether the amount of buffered thumbnail data 81 is equal to orlarger than the threshold for starting thumbnail data recording.

Since the amount of thumbnail data 81 is equal to or larger than thethreshold for starting thumbnail data recording at time t5, a seekoperation is performed or disk rotation is awaited based on thedetermination result during the period from time t5 to time t6, and attime t6, recording of the buffered thumbnail data 81 in the thumbnaildata recording area 162-1 is started.

As described with reference to FIG. 5, the thumbnail data 81 is recordedon the disk 45 in one cluster as the unit in which error correction iscarried out by the ECC. In this case, if the thumbnail data 81 issmaller than 12 kilobytes, any data string is added to the thumbnaildata 81 smaller than 12 kilobytes to increase the size to 12 kilobytes.

The thumbnail data recording area 162-1 is a contiguous area composed ofone or a plurality of clusters, and one or a plurality of items ofthumbnail data 81 is recorded in the thumbnail data recording area162-1.

Since recording of the thumbnail data 81 in the thumbnail data recordingarea 162-1 is ended at time t7, recording of the buffered MPEG2 systemstream in the stream unit 161-3 subsequent to the thumbnail datarecording area 162-1 is started.

Since recording of the MPEG2 system stream has reached the end of thestream unit 161-3 at time t8, a seek operation is performed or diskrotation is awaited during the period from time t8 to time t9, and attime t9, recording of the buffered MPEG2 system stream in the subsequentstream unit 161-4 is started.

Since the amount of buffered MPEG2 system stream is equal to or smallerthan one cluster at time t10, recording of the MPEG2 system stream inthe stream unit 161-4 is suspended, and a standby mode continues untilthe buffer is filled with the MPEG2 system stream.

It is assumed that the amount of buffered thumbnail data 81 is equal toor larger than the threshold for starting thumbnail data recording attime t11, at which recording the MPEG2 system stream in the stream unit161-4 is suspended. While recording the MPEG2 system stream issuspended, the recording apparatus does not monitor the amount ofthumbnail data 81.

At time t12, since the amount of buffered MPEG2 system stream is equalto or larger than the threshold for starting system stream recording,the suspended recording of the buffered MPEG2 system stream in thestream unit 161-4 is resumed.

When recording of the MPEG2 system stream has reached the end of thestream unit 161-4 at time t13, the recording apparatus determineswhether the amount of buffered thumbnail data 81 is equal to or largerthan the threshold for starting thumbnail data recording.

Since the amount of thumbnail data 81 is equal to or larger than thethreshold for starting thumbnail data recording at time t13, a seekoperation is performed or disk rotation is awaited based on thedetermination result during the period from time t13 to time t14, and attime t14, recording of the buffered thumbnail data 81 in the thumbnaildata recording area 162-2 is started.

Since recording of the thumbnail data 81 in the thumbnail data recordingarea 162-2 is ended at time t15, recording of the buffered MPEG2 systemstream in the stream unit 161-5 subsequent to the thumbnail datarecording area 162-2 is started.

When the thumbnail data 81 is to be recorded in the thumbnail datarecording area 162-2, the thumbnail data 81 smaller than 12 kilobytes ispadded with any data string, as in the thumbnail data recording area162-1, so that the size is increased to 12 kilobytes and then each itemof the thumbnail data 81 is recorded in one cluster. The thumbnail datarecording area 162-2 is a contiguous area composed of one or a pluralityof clusters, and one or a plurality of items of thumbnail data 81 isrecorded in the thumbnail data recording area 162-2.

At time t16, since recording of the MPEG2 system stream has reached theend of the stream unit 161-5, a seek operation is performed or diskrotation is awaited during the period from time t16 to time t17.Thereafter, recording of the buffered MPEG2 system stream in thesubsequent stream unit 161-6 is started at time t17.

Alternatively, recording may be performed at a position away from thestream unit 161 where the thumbnail data recording areas 162 areadjacent to each other.

FIG. 16 is a diagram depicting an example of thumbnail data recordingareas 162 in which adjacent recording is performed. Thumbnail datarecording area 162-1 to thumbnail data recording area 162-4 are providedadjacent to each other at a position away from the stream unit 161-1 tothe stream unit 161-(n+1).

By doing so, even when many items of thumbnail data 81 are read out fromthe disk 45, many items of thumbnail data 81 can be read out quicklyfrom the disk 45, without requiring a seek time or disk rotationallatency. Although this technique causes the number of seek operations orthe number of rotational delays in a case where the thumbnail data 81 isrecorded to become larger than the number of seek operations or thenumber of rotational delays in a case where the thumbnail data 81 is notrecorded, the thumbnail data 81 can be read out by sequentiallyaccessing a plurality of thumbnail data recording areas 162. Thus, thisrecording technique is suitable if the disk 45 is a disk having arelatively short access time (seek or rotational latency), such as ahard disk.

FIG. 17 is a diagram depicting the process of recording onto the disk 45in a case where the thumbnail data 81 are recorded together at aposition away from the stream unit 161. Also in this case, the buffermemory 43 is provided with a buffer for storing the MPEG2 system streamand a buffer for storing the thumbnail data 81 individually.

The upper graph in FIG. 17 depicts a time-lapse change in the amount ofbuffered MPEG2 system stream data, and the lower graph in FIG. 15depicts a time-lapse change in the amount of buffered thumbnail data 81.The vertical axis in FIG. 17 denotes the amount of data, and thehorizontal axis in FIG. 15 denotes time.

When time t31 is reached a predetermined time after recording has beenstarted, the amount of buffered MPEG2 system stream data is equal to orlarger than the threshold for starting system stream recording.Therefore, recording of the buffered MPEG2 system stream onto the disk45 as the stream unit 161-1 is started. At time t32, since recording ofthe MPEG2 system stream has reached the end of the stream unit 161-1, aseek operation is performed or disk rotation is awaited during theperiod from time t32 to time t33. Thereafter, recording of the bufferedMPEG2 system stream in the subsequent stream unit 161-2 is started attime t33.

It is assumed that the amount of buffered thumbnail data 81 is equal toor larger than the threshold for starting thumbnail data recording attime t34, at which the MPEG2 system stream is being recorded in thestream unit 161-2. While recording the MPEG2 system stream on the disk45, the recording apparatus does not monitor the amount of thumbnaildata 81.

Since recording of the MPEG2 system stream has reached the end of thestream unit 161-2 at time t35, the recording apparatus determines attime t35 whether the amount of buffered thumbnail data 81 is equal to orlarger than the threshold for starting thumbnail data recording.

Since the amount of thumbnail data 81 is equal to or larger than thethreshold for starting thumbnail data recording at time t35, a seekoperation is performed based on the determination result during theperiod from time t35 to time t36, and at time t36, recording of thebuffered thumbnail data 81 in the thumbnail data recording area 162-1 isstarted.

Also in this case, as described with reference to FIG. 5, the thumbnaildata 81 is recorded on the disk 45 in one cluster as the unit in whicherror correction is carried out by the ECC such that the cluster ispadded. The thumbnail data recording area 162-1 is a contiguous areacomposed of one or a plurality of clusters, and one or a plurality ofitems of thumbnail data 81 is recorded in the thumbnail data recordingarea 162-1.

Since recording of the thumbnail data 81 in the thumbnail data recordingarea 162-1 is ended at time t37, a seek operation is performed duringthe period from time t37 to time t38 and at time t38, recording of thebuffered MPEG2 system stream in the stream unit 161-3 at a position awayfrom the thumbnail data recording area 162-1 is started.

Since recording of the MPEG2 system stream has reached the end of thestream unit 161-3 at time t39, a seek operation is performed or diskrotation is awaited during the period from time t39 to time t40, and attime t40, recording of the buffered MPEG2 system stream in thesubsequent stream unit 161-4 is started.

Since the amount of buffered MPEG2 system stream is equal to or smallerthan one cluster at time t41, recording of the MPEG2 system stream inthe stream unit 161-4 is suspended, and a standby mode continues untilthe buffer is filled with the MPEG2 system stream.

It is assumed that the amount of buffered thumbnail data 81 is equal toor larger than the threshold for starting thumbnail data recording attime t42, at which recording the MPEG2 system stream in the stream unit161-4 is suspended. While recording the MPEG2 system stream issuspended, the recording apparatus does not monitor the amount ofthumbnail data 81.

At time t43, since the amount of buffered MPEG2 system stream is equalto or larger than the threshold for starting system stream recording,the suspended recording of the buffered MPEG2 system stream in thestream unit 161-4 is resumed.

When recording of the MPEG2 system stream has reached the end of thestream unit 161-4 at time t44, the recording apparatus determineswhether the amount of buffered thumbnail data 81 is equal to or largerthan the threshold for starting thumbnail data recording.

Since the amount of thumbnail data 81 is equal to or larger than thethreshold for starting thumbnail data recording at time t44, a seekoperation is performed based on the determination result during theperiod from time t44 to time t45, and at time t45, recording of thebuffered thumbnail data 81 in the thumbnail data recording area 162-2 isstarted.

Since recording of the thumbnail data 81 in the thumbnail data recordingarea 162-2 is ended at time t46, a seek operation is performed duringthe period from time t46 to time t47, and at time t47, recording of thebuffered MPEG2 system stream in the stream unit 161-5 at a position awayfrom the thumbnail data recording area 162-2 is started.

At time t48, since recording of the MPEG2 system stream has reached theend of the stream unit 161-5, a seek operation is performed or diskrotation is awaited during the period from time t48 to time t49.Thereafter, recording of the buffered MPEG2 system stream in thesubsequent stream unit 161-6 is started at time t49.

Processing by the recording apparatus will now be described withreference to a flowchart.

FIG. 18 is a flowchart illustrating the process of converting data bythe recording apparatus. In step S51, the moving-image compressionsection 38 compresses the acquired moving image in accordance with theMPEG2 technique. In step S52, the audio compression section 42compresses the acquired sound in accordance with the AC3 technique.

In step S53, from the moving image data stored in the buffer memory 35,the extraction section 51 of the number-of-pixels conversion section 40extracts one picture (frame) from one GOP of the moving image to becompressed in the moving-image compression section 38. For example, ifthe number of pictures (frames) constituting a GOP is predetermined inthe moving-image compression section 38, one picture (frame) isextracted for the number of pictures (frames). Furthermore, for example,the extraction section 51 may extract one picture (frame) from one GOPof the moving image based on a signal indicating the boundary betweenGOPs from the moving-image compression section 38.

In step S54, the number-of-pixels conversion section 40 converts thenumber of pixels of the extracted frame. In step S54, from among thepixels of the extracted frame, the number-of-pixels conversion section40 converts the number of pixels of the extracted frame by, for example,thinning out pixels at predetermined positions on the frame. Morespecifically, in step S54, the number-of-pixels conversion section 40calculates the mean value of the pixel values of four pixels, adjacentto one another, composed of two pixels in the vertical direction and twopixels in the horizontal direction from among the pixels of theextracted frame and then thins out three pixels from the four pixels bysetting the calculated mean value to one pixel replacing the four pixelsto convert the number of pixels of the frame.

In step S54, the number-of-pixels conversion section 40 can performconversion to produce a frame composed of any number of pixels and,therefore, the number of pixels of a resultant frame does not limit thepresent invention.

In step S55, the static-image compression section 41 compresses theframe whose number of pixels has been converted as a static image inaccordance with the JPEG technique to generate thumbnail data. Thestatic-image compression section 41 stores the generated thumbnail datain the buffer memory 43.

In step S56, the microcomputer 31 adjusts the file format of thethumbnail data acquired through compression. For example, in step S56,the microcomputer 31 can adjust the file format of the thumbnail dataacquired through compression to the PLF format, file format referred tofrom the file 101 in PLF format, static image package format, or thelocation-related data file 141 referred to from the track managementfile 131.

If the thumbnail data is adjusted to a file format referred to fromanother file, the microcomputer 31 generates a file for referring to thethumbnail data and the generated file is also recorded on the disk 45 asthumbnail data.

Although, in the above description, the processing from step S51 to stepS55 is carried out by the moving-image compression section 38 to theaudio compression section 42 and the extraction section 51, themicrocomputer 31 may carry out the processing from step S51 to step S55by executing the control program.

FIG. 19 is a flowchart for illustrating the process of recording data bythe microcomputer 31 for executing the control program. In step S71, thecontrol program acquires the amount of stored MPEG2 system stream datafrom the buffer memory 43 and determines whether the amount of MPEG2system stream data stored in the buffer memory 43 is equal to or largerthan a predetermined threshold for starting system stream recording.

If it is determine in step S71 that the amount of MPEG2 system streamdata stored in the buffer memory 43 is below the threshold for startingsystem stream recording, the procedure returns to step S71, where thedetermination processing is repeated until the amount of MPEG2 systemstream data is equal to or larger than the threshold for starting systemstream recording.

If it is determined in step S71 that the amount of the MPEG2 systemstream data stored in the buffer memory 43 is equal to or larger thanthe threshold for starting system stream recording, the flow proceeds tostep S72, where the control program causes the drive 44 to record ontothe disk one cluster of MPEG2 system stream stored in the buffer memory43.

In step S73, the control program determines whether recording of theMPEG2 system stream has reached the end of the stream unit. If it isdetermined that the recording of the MPEG2 system stream has not reachedthe end of the stream unit, the flow proceeds to step S74. In step S74,the control program determines whether the amount of MPEG2 system streamdata stored in the buffer memory 43 is below one cluster.

If it is determined in step S74 that the amount of MPEG2 system streamdata stored in the buffer memory 43 is not below one cluster, it meansthat the MPEG2 system stream can further be stored in the stream unit.Therefore, the flow returns to step S72, where the process of recordingthe MPEG2 system stream onto the cluster is repeated.

As a result of the processing from step S72 to step S74 being repeated,the MPEG2 system stream is recorded up to the end of the stream unit.

If it is determined in step S74 that the amount of MPEG2 system streamdata stored in the buffer memory 43 is below one cluster, no more of theMPEG2 system stream can be recorded in the stream unit. As a result, theflow returns to step S71 to continue a standby mode until the buffermemory 43 is filled with the MPEG2 system stream, and theabove-described processing is repeated.

When the MPEG2 system stream is recorded halfway in the stream unit andthe procedure returns to step S71 as a result of the determination instep S74, the MPEG2 system stream is recorded, in the processing of thesubsequent step S72, following the stream unit in which the MPEG2 systemstream has been recorded halfway.

On the other hand, if it is determined in step S73 that recording of theMPEG2 system stream has reached the end of the stream unit, the flowproceeds to step S75, where the control program determines whether theamount of thumbnail data is equal to or larger than a predeterminedthreshold for starting thumbnail data recording. If it is determined instep S75 that the amount of thumbnail data is equal to or larger thanthe threshold for starting thumbnail data recording, the flow proceedsto step S76, where the control program pads the thumbnail data such thatone item of thumbnail data is the same size as the unit in which errorcorrection is carried out by the ECC, for example, 12 kilobytes.

In step S77, the control program causes the drive 44 to record onepadded item of thumbnail data in one cluster of the disk 45.

In step S78, the control program determines whether there is no morethumbnail data stored in the buffer memory 43. If it is determined thatthere remains thumbnail data stored in the buffer memory 43, the flowreturns to step S76 to further record the thumbnail data in thethumbnail data recording area 162, which is a contiguous area, and theprocess of recording the thumbnail data is repeated.

If it is determined in step S78 that there is no more thumbnail datastored in the buffer memory 43, no more of the thumbnail data can berecorded. Therefore, the flow proceeds to step S74, where the process ofdetermining whether the MPEG2 system stream which can be recorded on thedisk 45 is stored in the buffer memory 43 is carried out and theabove-described processing is repeated.

If it is determined in step S75 that the amount of thumbnail data isbelow the threshold for starting thumbnail data recording, it is notnecessary to record thumbnail data on the disk 45, and the flow proceedsto step S74, where the process of determining whether the MPEG2 systemstream which can be recorded on the disk 45 is stored in the buffermemory 43 is carried out and the above-described processing is repeated.

As described above, with the recording apparatus, thumbnail datacorresponding to the frame extracted from a unit composed of a pluralityof frames (pictures), i.e., the unit in which a moving image is encoded,is recorded on the disk 45 in association with the extracted unit.

Next, a playback apparatus for reading out thumbnail data from the disk45 on which thumbnail data corresponding to the frame extracted from aunit composed of a plurality of frames (pictures), i.e., the unit inwhich a moving image is encoded, is recorded will be described.

The playback apparatus for reading out thumbnail data from the disk 45can be realized as a recording and playback apparatus including thefunctions correspond to the recording apparatus whose structure has beendescribed with reference to FIG. 3.

FIG. 20 is a block diagram depicting a structure of one embodiment of aplayback block in a recording and playback apparatus according to thepresent invention, where the recording and playback apparatus includesthe functions of the recording apparatus whose structure has beendescribed with reference to FIG. 3. The same components as those shownin FIG. 3 are denoted by the same symbols, and thus a descriptionthereof will be omitted.

The microcomputer 31 executes the predetermined control program to issuea command for operation to each section of the playback block in therecording and playback apparatus based on a signal from the playbackstart/stop button 201 according to an operation of the user.

Under the control of the microcomputer 31, the drive 44 reads out anMPEG2 system stream and thumbnail data from the mounted disk 45. Thedrive 44 stores the read-out MPEG2 system stream and thumbnail data inthe buffer memory 43.

A demultiplexer 202 separates moving image data and audio datamultiplexed in the MPEG2 system stream stored in the buffer memory 43,supplies the separated moving image data to a moving-image decompressionsection 203, and furthermore, supplies the separated audio data to anaudio decompression section 205.

Under the control of the microcomputer 31, the moving-imagedecompression section 203 performs decompression by decoding the movingimage data supplied from the demultiplexer 202, which is compressed andencoded through a predetermined technique for compressing and encodingmoving image data, and supplies the decoded moving image data to thebuffer memory 35. For example, the moving-image decompression section203 decodes the moving image data compressed and encoded in accordancewith the MPEG2 technique and supplies the decoded moving image data(so-called, baseband moving image data) to the buffer memory 35.

Under the control of the microcomputer 31, a static-image decompressionsection 204 acquires thumbnail data from the buffer memory 43 storingthe thumbnail data, performs decompression by decoding the thumbnaildata compressed and encoded in accordance with a predetermined techniquefor compressing and encoding static image data, and supplies the decodedthumbnail data to the buffer memory 35. For example, the static-imagedecompression section 204 decodes thumbnail data compressed and encodedin accordance with the JPEG technique and supplies the decoded thumbnaildata to the buffer memory 35.

Under the control of the microcomputer 31, the audio decompressionsection 205 performs decompression by decoding the audio data suppliedfrom the demultiplexer 202, which is compressed and encoded through apredetermined technique for compressing and encoding audio data, andsupplies the decoded audio data to the buffer memory 35. For example,the audio decompression section 205 decodes audio data compressed andencoded in accordance with the AC3 technique and supplies the decodedaudio data to the buffer memory 35.

An image output interface 206 is an interface between the buffer memory35 and a display section 207, performs, for example, parallel-to-serialconversion of the image data supplied from the buffer memory 35 toconvert the image data into image data (image signal) in predeterminedformat that can be used by the display section 207, and supplies theresultant image data to the display section 207. The image outputinterface 206 controls display of the image in the display section 207.

The display section 207 includes, for example, a liquid crystal displayunit or an organic EL (Electro Luminescence) display unit and displaysmoving images and static images based on image data supplied through theimage output interface 206.

An audio output interface 208 is an interface between the buffer memory35 and an audio output section 209, performs, for example,parallel-to-serial conversion or digital-to-analog conversion of theaudio data supplied from the buffer memory 35 to convert the audio datainto audio data (audio signal) in predetermined format that can be usedby the audio output section 209, and supplies the resultant audio data(audio signal) to the audio output section 209.

The audio output section 209 includes an audio amplifier or aloudspeaker and outputs sound based on audio data (audio signal)supplied through the audio output interface 208.

FIG. 21 is a flowchart illustrating the process of locating anddisplaying an edit point by the microcomputer 31 for executing thecontrol program, the static-image decompression section 204, and thedisplay section 207.

In step S101, the control program causes the drive 44 to read out amanagement information file from the disk 45. For example, in step S101,the control program causes the drive 44 to read out from the disk 45 thefile 101 in PLF format for referring to the thumbnail data 81 stored inan external file or the track management file 131, serving as amanagement information file, shown in FIG. 10. The drive 44 stores theread-out management information file in the buffer memory 43.

In step S102, the control program causes the drive 44 to read out fromthe disk 45 as much thumbnail data 81 as can be stored in the buffermemory 43, starting with the first item of thumbnail data 81, based onthe management information file acquired from the buffer memory 43.

In step S103, the control program causes the drive 44 to store theread-out thumbnail data 81 in the buffer memory 43. In step S104, thestatic-image decompression section 204 acquires the thumbnail data 81from the buffer memory 43 storing the thumbnail data 81 and decompressesthe acquired thumbnail data 81. For example, the static-imagedecompression section 204 performs decompression by decoding thethumbnail data 81 compressed and encoded in accordance with the JPEGtechnique.

In step S105, the static-image decompression section 204 stores thedecompressed thumbnail data 81 in the buffer memory 35. In step S106,the image output interface 206 controls display of the image in thedisplay section 207 based on the thumbnail data 81 acquired from thebuffer memory 35 to allow the display section 207 to display the image.

In step S107, the control program determines whether an instruction forshifting to the subsequent point has been issued based on a signal fromthe playback start/stop button 201 according to an operation of theuser. If it is determined in step S107 that no instruction for shiftingto the subsequent point has been issued, displaying the image iscontinued and the flow returns to step S107 to repeat the determinationprocessing.

If it is determined in step S107 that an instruction for shifting to thesubsequent point has been issued, the flow proceeds to step S108, wherethe control program identifies the thumbnail data 81 corresponding tothe GOP at the specified point based on the read-out managementinformation file.

In step S109, the control program determines whether the identifiedthumbnail data 81 is stored in the buffer memory 43. If it is determinedin step S109 that the identified thumbnail data 81 is not stored in thebuffer memory 43, the flow proceeds to step S110, where the controlprogram causes the drive 44 to read out from the disk 45 as muchthumbnail data 81 as can be stored in the buffer memory 43, startingwith the identified thumbnail data, based on the management informationfile acquired from the buffer memory 43.

In step S110, the control program can cause the drive 44 to read out aplurality of items of thumbnail data 81 from one item of thumbnail datarecording area 162 on the disk 45 through one processing operation.

As described with reference to FIG. 5, since the thumbnail data 81increased to 12 kilobytes as a result of any data string being added isrecorded on the disk 45 in one cluster as a unit in which errorcorrection is carried out by the ECC, the control program can calculatethe physical address of the cluster in which the thumbnail data 81 to beread out subsequently is recorded through a simple arithmetic operation.As a result, the thumbnail data 81 can be read out more quickly.

In step S111, the control program causes the drive 44 to store theread-out thumbnail data 81 in the buffer memory 43, and the procedureproceeds to step S112.

If it is determined in step S109 that the identified thumbnail data 81is stored in the buffer memory 43, it is not necessary to read thethumbnail data 81 from the disk 45, and therefore, the processing fromstep S110 and step S111 is skipped and the procedure proceeds to stepS112.

In step S112, the static-image decompression section 204 acquires thethumbnail data 81 from the buffer memory 43 storing the thumbnail data81 and decompresses the acquired thumbnail data 81. For example, thestatic-image decompression section 204 performs decompression bydecoding the thumbnail data 81 compressed and encoded in accordance withthe JPEG technique.

In step S113, the static-image decompression section 204 stores thedecompressed thumbnail data 81 in the buffer memory 35. In step S114,the display section 207 displays an image based on the thumbnail data 81acquired from the buffer memory 35 through the image output interface206.

The procedure returns to step S107, where the processing for readingfrom the disk 45 the thumbnail data corresponding to a GOP at the pointspecified according to an instruction from the user, decoding thethumbnail data 81, and displaying an image is repeated.

As described above, if only the thumbnail data 81 is read out from thedisk 45, the desired thumbnail can be displayed more quickly.Furthermore, the thumbnail data 81 can be read out from the disk 45together with the MPEG2 system stream.

For example, in the processing of step S114, the display section 207 maydisplay a moving image 231 on the entire screen and, furthermore,display a thumbnail 232 in a partial area of the screen, as shown inFIG. 22. In this case, for example, in the processing of step S114, thedisplay section 207 displays the moving image 231 so that it is playedback at a normal speed and furthermore, displays the thumbnail 232corresponding to the GOP at the specified point in the partial area ofthe screen.

For example, if an instruction for a fast-forward or rewind operation isissued from the user, the display section 207 displays the moving image231 so that it is played back at a normal speed and furthermore,displays the thumbnail 232 being fast-forwarded or rewound in thepartial area of the screen.

In response to an instruction from the user, the playback apparatus candisplay on the display section 207 the moving image 231 from the GOPcorresponding to the displayed thumbnail 232.

By doing so, the user can quickly perceive the overview of the movingimage with the help of the thumbnail 232 corresponding to the GOP at thespecified point and furthermore, can view detailed contents of themoving image due to the displayed moving image. As a result, locatingthe beginning of the moving image at a desired position or positioningan edit point can be quickly carried out.

The process of simultaneously reading out the thumbnail data 81 and themoving image data recorded on the disk 45 by means of the microcomputer31 executing the control program will be described below.

The control program causes the drive 44 to read out from the disk 45 thetrack management file 131, serving as a management information file ofthe MPEG2 system stream, and the file management information of the filesystem. The control program causes the drive 44 to read out from thedisk 45 the MPEG2 system stream in units of the stream unit based on thetrack management file 131, serving as the management information file ofthe MPEG2 system stream, and the file management information of the filesystem.

In this case, the control program allows the drive 44 to continue toread the MPEG2 system stream until reading the MPEG2 system stream fromone stream unit 161 is completed. When reading the MPEG2 system streamfrom the one stream unit 161 is completed, the amount of MPEG2 systemstream data stored in the buffer memory 43 is calculated.

The control program acquires the amount of MPEG2 system stream data tobe read out subsequently and the playback time of the moving image basedon the file in track management file format, serving as the managementinformation file of the MPEG2 system stream, and the file managementinformation of the file system. The control program acquires thephysical address of the stream unit 161 in which the MPEG2 system streamto be read out subsequently is recorded and the physical address of thethumbnail data recording area 162 in which the thumbnail data 81 to beread out subsequently is recorded based on the track management file131, serving as the management information file of the MPEG2 systemstream, and the file management information of file system.

The control program calculate the estimated amount of MPEG2 systemstream data to be stored in the buffer memory 43 at the time when theMPEG2 system stream is read out from the subsequent stream unit 161, ina case where the control program reads out the MPEG2 system stream fromthe subsequent stream unit 161 and stores it in the buffer memory 43,based on the amount of MPEG2 system stream data stored in the currentbuffer memory 43, the track management file 131, and the file managementinformation of the file system. The control program determines whetherthe calculated and estimated amount of MPEG2 system stream data exceedsthe upper limit value of the capacity of the buffer memory 43.

If it is determined that the estimated amount of MPEG2 system streamdata exceeds the upper limit value of the capacity of the buffer memory43, the control program causes the drive 44 to suspend reading the MPEG2system stream from the stream unit 161 on the disk 45 and causes thedrive 44 to read out the thumbnail data 81 from the thumbnail datarecording area 162 on the disk 45. Thereafter, when reading thethumbnail data 81 from the thumbnail data recording area is ended, thecontrol program causes the drive 44 to read out the MPEG2 system streamfrom the stream unit 161 on the disk 45.

On the other hand, if it is determined that the estimated amount ofMPEG2 system stream data does not exceed the upper limit value of thecapacity of the buffer memory 43, the control program causes the drive44 to read out the MPEG2 system stream from the stream unit 161 on thedisk 45.

As described above, the amount of MPEG2 system stream data expected tobe stored in the buffer memory 43 at the time when reading the streamunit 161 of the AMPEG2 system stream to be read out subsequently isended is obtained at the end of the stream unit 161, and data to be readout subsequently is determined according to the estimated amount ofdata. Therefore, the number of access operations by the drive 44 isreduced and the amount of data to be read out from the disk 45 per unittime increases, thus increasing the data reading efficiency.

As shown in FIG. 14, when the thumbnail data 81 recorded in thethumbnail data recording area 162-1 and the thumbnail data recordingarea 162-2 is to be read out in a case where the stream unit 161-1 tothe stream unit 161-6, as well as the thumbnail data recording area162-1 and the thumbnail data recording area 162-2, are recorded on thedisk 45 such that the thumbnail data recording area 162-1 is adjacent tothe stream unit 161-3 and the thumbnail data recording area 162-2 isadjacent to the stream unit 161-6, the thumbnail data 81 recorded in thethumbnail data recording area 162-1, which is a contiguous area, issequentially read out, and then a head (not shown) of the playbackapparatus moves to the thumbnail data recording area 162-2 through aseek operation or a rotational delay of the disk 45 to sequentially readout the thumbnail data 81 recorded in the thumbnail data recording area162-2, which is a contiguous area, as shown in FIG. 23.

Therefore, the thumbnail data 81 can be read out from the disk 45 veryquickly, compared with a case where the stream unit 161-1 to the streamunit 161-6 are sequentially read out.

Furthermore, as shown in FIG. 16, when the thumbnail data 81 recorded inthe thumbnail data recording area 162-1 to thumbnail data recording area162-4 is to be read out in a case where the stream unit 161-1 to thestream unit 161-(n+1), as well as the thumbnail data recording area162-1 to thumbnail data recording area 162-4, are recorded on the disk45 such that the thumbnail data recording area 162-1 to thumbnail datarecording area 162-4 are recorded adjacent to one another at a positionaway from stream unit 161-1 to stream unit 161-(n+1), the thumbnail data81 recorded in the thumbnail data recording area 162-1 to the thumbnaildata recording area 162-4 as a contiguous area is sequentially read outwithout causing a seek operation or rotational delay of the disk 45, asshown in FIG. 24.

Therefore, the thumbnail data 81 can be read out from the disk 45 evenmore quickly.

FIG. 25 is a diagram depicting a change in the amount of thumbnail data81 stored in the buffer memory 43 for a fast-forward operation ofthumbnails. The vertical axis in FIG. 25 denotes the amount of data, andthe horizontal axis in FIG. 25 denotes time.

Here, the amount of thumbnail data 81 stored in the buffer memory 43means the amount of data from the thumbnail data 81 for displaying thethumbnail subsequent to the currently displayed thumbnail to thethumbnail data 81 for displaying the rearmost (latest) thumbnail in timeon the image.

When fast-forward processing is started, the drive 44 reads out thethumbnail data 81 from the disk 45 and stores it in the buffer memory43. When the amount of thumbnail data 81 stored in the buffer memory 43is equal to or larger than the threshold for starting thumbnail display,the process of displaying the thumbnails is started and the thumbnaildata 81 is sequentially read out from the buffer memory 43.

The amount of thumbnail data 81 for displaying one thumbnail is smallcompared with the amount of data of the moving image. Furthermore, sinceone thumbnail corresponds to one GOP, the amount of thumbnail data 81per unit time on the image is even smaller. Even after the process ofdisplaying the thumbnails is started, the amount of thumbnail data 81read out from the disk 45 for storage is large compared with the amountof thumbnail data 81 used for display.

Therefore, even after the process of displaying thumbnails is started,the amount of thumbnail data 81 stored in the buffer memory 43 increasesover time as long as the thumbnail data 81 is read out from the disk 45.

For example, at time t101, when the amount of thumbnail data 81 storedin the buffer memory 43 is equal to the buffer upper limit valueindicating the amount of data that can be stored in the buffer memory43, the drive 44 suspends (terminates) reading the thumbnail data 81from the disk 45 at time t101.

When the drive 44 suspends (terminates) reading the thumbnail data 81from the disk 45, the amount of thumbnail data 81 stored in the buffermemory 43 decreases as the process of displaying the thumbnailsproceeds.

For example, when display of the thumbnails is paused during the periodfrom time t103 to time t104, the amount of thumbnail data 81 stored inthe buffer memory 43 does not change during this period.

At time t104, if faster forward processing is specified, the amount ofthumbnail data 81 used per unit time increases, and therefore, theamount of thumbnail data 81 stored in the buffer memory 43 decreasesmore quickly.

At time t105, when the amount of thumbnail data 81 stored in the buffermemory 43 reaches the threshold for resuming thumbnail data reading, thedrive 44 starts reading the thumbnail data 81 from the disk 45. Duringthe period from time t105 to time t106, when a seek operation is carriedout, rotation of the disk 45 is awaited, and the head (not shown)reaches the position at which the thumbnail data 81 to be read out isrecorded at time t106, the drive 44 starts reading the thumbnail data 81from the disk 45 and stores the read-out thumbnail data 81 in the buffermemory 43.

For example, at time t107, when the amount of thumbnail data 81 storedin the buffer memory 43 is equal to the buffer upper limit valueindicating the amount of data that can be stored in the buffer memory43, the drive 44 suspends (terminates) reading the thumbnail data 81from the disk 45.

For example, if display of the thumbnails is paused from time t108, theamount of thumbnail data 81 stored in the buffer memory 43 does notchange thereafter.

FIG. 26 is a diagram depicting a change in the amount of thumbnail data81 stored in the buffer memory 43 during a rewind operation of thethumbnail. The vertical axis in FIG. 26 denotes the amount of data, andthe horizontal axis in FIG. 26 denotes time.

Here, the amount of thumbnail data 81 stored in the buffer memory 43means the amount of data from the thumbnail data 81 for displaying thethumbnail previous to the currently displayed thumbnail to the thumbnaildata 81 for displaying the frontmost (earliest) thumbnail in time on theimage.

When rewind processing is started, the drive 44 reads out the thumbnaildata 81 from the disk 45 and stores it in the buffer memory 43. When theamount of thumbnail data 81 stored in the buffer memory 43 is equal toor larger than the threshold for starting thumbnail display, the processof displaying the thumbnails is started and the thumbnail data 81 issequentially read out from the buffer memory 43.

Here, when a rewind operation is performed, the thumbnails are displayedin the direction opposite to the direction in which time on the imagepasses, and therefore, the thumbnail data 81 is also required in thedirection opposite to the direction in which time on the image passes.

As shown in FIG. 27, in a case where the thumbnail data 81 of thefrontmost thumbnail in time on the image is recorded in the thumbnaildata recording area 162-1; the thumbnail data 81 of the thumbnailsubsequent to the thumbnail of the thumbnail data 81 recorded in thethumbnail data recording area 162-1 is recorded in the thumbnail datarecording area 162-2 in time on the image; and the thumbnail data 81 ofthe thumbnail subsequent to the thumbnail of the thumbnail data 81recorded in the thumbnail data recording area 162-2 is recorded in thethumbnail data recording area in time on the image, the drive 44 readsout the thumbnail data 81 from the thumbnail data recording area on thedisk 45, performs a seek operation to the thumbnail data recording area162-2, and reads out the thumbnail data 81 from the thumbnail datarecording area when the head reaches the thumbnail data recording area162-2. Furthermore, the drive 44 performs a seek operation up to thethumbnail data recording area 162-1 and reads out the thumbnail data 81from the thumbnail data recording area 162-1 when the head reaches thethumbnail data recording area 162-1.

Therefore, as shown in FIG. 28, when the thumbnail data 81 is read outfrom the thumbnail data recording area 162-3 during the period from timet141 to time t142 and the read out thumbnail data 81 is supplied to thebuffer memory 43 at time t142, a constant amount of read-out thumbnaildata 81 is stored in the buffer memory 43 at time t142. When thethumbnail data 81 is read out from the thumbnail data recording area162-2 during the period from time t142 to time t143 and the read outthumbnail data 81 is supplied to the buffer memory 43 at time t143, aconstant amount of read-out thumbnail data 81 is stored in the buffermemory 43 at time t143. Likewise, when the thumbnail data 81 is read outfrom the thumbnail data recording area 162-1 during the period from timet143 to time t144 and the read out thumbnail data 81 is supplied to thebuffer memory 43 at time t144, a constant amount of read-out thumbnaildata 81 is stored in the buffer memory 43 at time t144.

As described above, during a rewind operation, the amount of thumbnaildata 81 stored in the buffer memory 43 increases in a staircase patternover time.

Also during a rewind operation, the amount of thumbnail data 81 read outfrom the disk 45 is large compared with the amount of thumbnail data 81used for display.

Therefore, even after the process of displaying thumbnails is started,the amount of thumbnail data 81 stored in the buffer memory 43 increasesover time as long as the thumbnail data 81 is read out from the disk 45.

For example, at time t121, when the amount of thumbnail data 81 storedin the buffer memory 43 is equal to the buffer upper limit valueindicating the amount of data that can be stored in the buffer memory43, the drive 44 suspends (terminates) reading the thumbnail data 81from the disk 45 at time t121.

When the drive 44 suspends (terminates) reading the thumbnail data 81from the disk 45, the amount of thumbnail data 81 stored in the buffermemory 43 decreases as the process of displaying the thumbnailsproceeds.

For example, when display of the thumbnails is paused during the periodfrom time t123 to time t124, the amount of thumbnail data 81 stored inthe buffer memory 43 does not change during this period.

At time t124, if faster rewind processing is specified, the amount ofthumbnail data 81 used per unit time increases, and therefore, theamount of thumbnail data 81 stored in the buffer memory 43 decreasesmore quickly.

At time t125, when the amount of thumbnail data 81 stored in the buffermemory 43 reaches the threshold for resuming thumbnail data reading, thedrive 44 starts reading the thumbnail data 81 from the disk 45. Duringthe period from time t125 to time t126, when a seek operation is carriedout, rotation of the disk 45 is awaited, and the head (not shown)reaches the position at which the thumbnail data 81 to be read out isrecorded at time t126, the drive 44 starts reading the thumbnail data 81from the disk 45 and stores the read-out thumbnail data 81 in the buffermemory 43.

For example, at time t127, when the amount of thumbnail data 81 storedin the buffer memory 43 is equal to the buffer upper limit valueindicating the amount of data that can be stored in the buffer memory43, the drive 44 suspends (terminates) reading the thumbnail data 81from the disk 45.

For example, if display of the thumbnails is paused from time t128, theamount of thumbnail data 81 stored in the buffer memory 43 does notchange thereafter.

FIG. 29 is a flowchart for illustrating the process of reading thumbnaildata by the microcomputer 31 for executing the control program. In stepS141, the control program causes the drive 44 to read the thumbnail data81 from the disk 45. The drive 44 stores the read-out thumbnail data 81in the buffer memory 43.

In step S142, the control program determines whether the amount ofthumbnail data 81 stored in the buffer memory 43 is equal to or largerthan the threshold for starting thumbnail display.

Here, in the case of fast-forward or normal playback, the amount ofthumbnail data 81 stored in the buffer memory 43 means the amount ofdata from the thumbnail data 81 for displaying the thumbnail subsequentto the currently displayed thumbnail to the thumbnail data 81 fordisplaying the rearmost (latest) thumbnail in time on the image. In thecase of rewind processing, the amount of thumbnail data 81 stored in thebuffer memory 43 means the amount of data from the thumbnail data 81 fordisplaying the thumbnail previous to the currently displayed thumbnailto the thumbnail data 81 for displaying the frontmost (earliest)thumbnail in time on the image.

If it is determined in step S142 that the amount of thumbnail data 81stored in the buffer memory 43 is below the threshold for startingthumbnail display, the flow proceeds to step S141, where the process ofreading out the thumbnail data is repeated.

If it is determined in step S142 that the amount of thumbnail datastored in the buffer memory 43 is equal to or larger than the thresholdfor starting thumbnail display, the static-image decompression section204 performs decompression by decoding the thumbnail data 81 stored inthe buffer memory 43 and displays the thumbnail on the display section207.

In step S144, the control program causes the drive 44 to read out thethumbnail data 81 from the disk 45. The drive 44 stores the read-outthumbnail data 81 in the buffer memory 43.

In step S145, the control program determines whether the buffer memory43 is full. In other words, in step S145, the control program determineswhether the amount of thumbnail data 81 stored in the buffer memory 43has reached (is equal to) the buffer memory upper limit value.

If it is determined in step S145 that the buffer memory is not full, theprocedure returns to step S144, where the process of reading thethumbnail data 81 is repeated.

If it is determined in step S145 that the buffer memory 43 is full, thebuffer memory 43 cannot store any more thumbnail data 81. Therefore, theflow proceeds to step S146 without reading the thumbnail data 81, andthe control program determines whether the thumbnails are displayed inthe order of playing back the moving image, namely, in the order offast-forward or normal playback operation.

If it is determined in step S146 that the thumbnails are displayed inthe order of playing back the moving image, in other words, since theapparatus is in a fast-forward or normal playback mode, the flowproceeds to step S147, where from among the thumbnail data 81 stored inthe buffer memory 43, the control program calculates the amount ofthumbnail data 81 from the thumbnail subsequent to the currentlydisplayed thumbnail to the rearmost (latest) thumbnail in time on theimage. Thereafter, the flow proceeds to step S149.

If it is determined in step S146 that the thumbnails are not displayedin the order of playing back the moving image, in other words, since theapparatus is in a rewind playback mode, the flow proceeds to step S148,where from among the thumbnail data 81 stored in the buffer memory 43,the control program calculates the amount of thumbnail data 81 from thefrontmost (earliest) thumbnail in time on the image to the thumbnailprevious to the currently displayed thumbnail. Thereafter, the flowproceeds to step S149.

In step S149, the control program determines whether the calculatedamount of data is equal to or smaller than the threshold for resumingthumbnail data reading. If it is determined in step S149 that thecalculated amount of data is not equal to or below the threshold forresuming thumbnail data reading, it is not necessary read out thethumbnail data 81. Therefore, the flow returns to step S146, where theabove-described processing is repeated.

If it is determined in step S149 that the calculated amount of data isequal to or smaller than the threshold for resuming thumbnail datareading, it is necessary to read out the thumbnail data 81. Therefore,the flow returns to step S144, where the process of reading thethumbnail data 81 is carried out.

The foregoing processing is described with reference to FIG. 30 to FIG.34. If display of the thumbnail at time t0 on the image is directedduring a fast-forward operation, the control program instructs the drive44 to start reading with the thumbnail data 81 at time (t0−T1) apredetermined time T1 back from time t0 on the image.

Hereinafter, the thumbnail data 81 at time t is referred to as thethumbnail data t, as appropriate.

Reading is started with the thumbnail data (t0−T1) at time (t0−T1) apredetermined time T1 back from time t0 on the image in order to quicklystart a rewind operation by responding to any instruction for a rewindoperation.

The drive 44 reads out from the disk 45 the section from the thumbnaildata (t0−T1) to the thumbnail data t0 at time t0 on the image, stores itin the buffer memory 43 in the form of a ring buffer, reads out thethumbnail data (t0+T1) from the thumbnail data t0 to time (t0+T1) thepredetermined time T1 ahead from time t0, and stores it in the buffermemory 43 in the form of a ring buffer. For the sake of convenience,symbol A in FIG. 30 indicates the thumbnail data 81 stored in the buffermemory 43 corresponding to the section from time (t0−T1) on the image totime t0 on the image. For the sake of convenience, symbol B in FIG. 30indicates the thumbnail data 81 stored in the buffer memory 43corresponding to the section from time t0 on the image to time (t0+T1)on the image.

Thereafter, the drive 44 reads out the thumbnail data 81 from the disk45 until the buffer memory 43 in the form of a ring buffer becomes fulland stores the thumbnail data 81 in the buffer memory 43. For the sakeof convenience, symbol C in FIG. 30 indicates the thumbnail data 81stored in the buffer memory 43 corresponding to the section from time(t0+T1) on the image to time (t0+Tn) on the image.

When a fast-forward operation is carried out and the thumbnail at timetn on the image is displayed, the thumbnail data 81 corresponding to thesection from time t0 on the image to time tn on the image is used up(cannot be used for the subsequent fast-forward processing), as shown inFIG. 31. Therefore, the thumbnail data 81, stored in the buffer memory43, that can be used for the subsequent fast-forward processing is, forexample, only the thumbnail data (t(n+1)) to the thumbnail data (tn+T1).For the sake of convenience, symbol D in FIG. 30 indicates the thumbnaildata 81 stored in the buffer memory 43 corresponding to the section fromtime tn on the image to time (tn+T1) on the image.

When the amount of thumbnail data 81 that can be used for the subsequentfast-forward processing is equal to or smaller than the threshold forresuming thumbnail data reading, as shown in FIG. 32, the drive 44sequentially reads out from the disk 45 the thumbnail data 81 subsequentto the thumbnail data (tn+T1) and stores the read-out thumbnail data 81in the buffer memory 43 until the buffer memory 43 in the form of a ringbuffer is full, while preventing the section corresponding to thethumbnail data 81 from time (tn−T1) on the image to time (tn+T1) on theimage from being overwritten. For the sake of convenience, symbol E inFIG. 30 indicates the thumbnail data 81 stored in the buffer memory 43corresponding to time on the image from time (tn+T1+1) on the image.

On the other hand, when the thumbnail at time tm on the image isdisplayed during a rewind operation and the thumbnail data 81corresponding to the section up to time tm on the image is used up, asshown in FIG. 33, the thumbnail data 81, stored in the buffer memory 43,that can be used for the subsequent rewind processing is, for example,only the thumbnail data (t(m−1)) to the thumbnail data (tm−T1). For thesake of convenience, symbol E in FIG. 30 indicates the thumbnail data 81stored in the buffer memory 43 corresponding to the section from time tmon the image to time (tm−T1) on the image.

When the amount of thumbnail data 81 that can be used for the subsequentrewind processing is equal to or smaller than the threshold for resumingthumbnail data reading, as shown in FIG. 34, the drive 44 reads out fromthe disk 45 the thumbnail data 81 in the reverse direction, startingwith the thumbnail data (tm−T1−1), and stores the read-out thumbnaildata 81 in the buffer memory 43 until the buffer memory 43 in the formof a ring buffer is full, while preventing the section corresponding tothe thumbnail data 81 from time (tm+T1) on the image to time (tm−T1) onthe image from being overwritten.

In this case, since the thumbnail data 81 is read out on the basis ofthe thumbnail recording area 162, the thumbnail data 81, indicated by Fin FIG. 30, including the thumbnail data (tm−T1−1) is stored in thebuffer memory 43 in the order indicated by the thick arrows in FIG. 34,and then the thumbnail data 81, indicated by G followed by the thumbnaildata 81 indicated by F in FIG. 30, is stored in the buffer memory 43. Inthe same manner, the thumbnail data 81 is read out from the disk 45 andstored in the buffer memory 43 going backward in time on the image.

A recording apparatus in a case where thumbnails corresponding to movingimage data recorded on the disk 45 are generated and then recorded onthe disk 45 will now be described.

FIG. 35 is a block diagram depicting a structure of one embodiment of arecording and playback apparatus, according to the present invention,for reading out an MPEG2 program stream from the disk 45 on which theMPEG2 program stream is recorded and generating thumbnail datacorresponding to the MPEG2 program stream to record it on the disk 45.The same components as those shown in FIG. 20 are denoted by the samesymbols, and thus a description thereof will be omitted.

The microcomputer 31 executes the predetermined control program to issuea command for operation to each section of the recording and playbackapparatus based on a signal from a start/stop button 301 according to anoperation of the user.

The drive 44 reads out a program stream in accordance with the MPEG2technique from the disk 45 and stores the read-out program stream inaccordance with the MPEG2 technique in the buffer memory 43. Thedemultiplexer 202 separates the MPEG2 video elementary stream as imagedata in accordance with the MPEG2 technique and audio data from theprogram stream in accordance with the MPEG2 technique stored in thebuffer memory 43.

Under the control of the microcomputer 31, an I-picture selection anddecoding section 302 selects an I (intra) picture from among the MPEG2video elementary stream separated by the demultiplexer 202 and decodesthe selected I picture. The I-picture selection and decoding section 302supplies the decoded picture to a resolution conversion section 303.

As with the number-of-pixels conversion section 40, the resolutionconversion section 303 converts the resolution of the decoded pictureunder the control of the microcomputer 31. For example, the resolutionconversion section 303 converts the resolution of the extracted pictureby thinning out pixels from the picture.

The resolution conversion section 303 supplies the image data whosenumber of pixels has been converted to a JPEG encoding section 304.Under the control of the microcomputer 31, the JPEG encoding section 304encodes the image data supplied from the resolution conversion section303 in accordance with the JPEG technique. The JPEG encoding section 304supplies the image data encoded in accordance with the JPEG technique toa file-format conversion section 305 as thumbnail data.

The file-format conversion section 305 converts the file format of thethumbnail data into the PLF format, file format referred to from thefile 101 in PLF format, static image package format, or location-relateddata file format referred to from the track management file 131. Thefile-format conversion section 305 supplies the thumbnail data withconverted file format to the buffer memory 43.

Here, for all GOPs of moving image data for which a command for creatingthumbnails has been issued, the file-format conversion section 305 mayconvert the file formats of all items of thumbnail data at a time intoone file or a predetermined number of files for all items of thumbnaildata after the thumbnail data has been supplied by the resolutionconversion section 303.

The drive 44 records on the disk 45 the thumbnail data which has beenconverted into a predetermined file format and is stored in the buffermemory 43. If the file-format conversion section 305 converts the fileformats of all items of thumbnail data at a time into one file or apredetermined number of files for all items of thumbnail data after thethumbnail data has been supplied by the resolution conversion section303 for all GOPs of moving image data for which a command for creatingthumbnails has been issued, the drive 44 records on the disk 45 thethumbnail data whose file formats have been converted all at a time.

FIG. 36 is a block diagram depicting the structure of the I-pictureselection and decoding section 302. A buffer 321 temporarily stores themoving image data in accordance with the MPEG2 technique, which is anMPEG2 video elementary stream, supplied from the demultiplexer 202. AnI-picture determination section 322 determines whether each of thepictures constituting the moving image data in accordance with the MPEG2technique stored in the buffer 321 is an I picture by, for example,referring to the picture coding type in the picture header.

A selector 323 supplies data of the picture stored in the buffer 321 toa variable-length code decoder 324 or controls the supply of the data ofpicture stored in the buffer 321 to the variable-length code decoder 324based on a signal indicating whether the picture is an I picturesupplied from the I-picture determination section 322. Morespecifically, if a signal indicating that the picture is an I picture issupplied from the I-picture determination section 322, the selector 323supplies data of the picture, which is an I picture stored in the buffer321, to the variable-length code decoder 324. If a signal indicatingthat the picture is not an I picture is supplied from the I-picturedetermination section 322, the selector 323 controls the supply of dataof the picture, which is a B picture or a P picture stored in the buffer321, to the variable-length code decoder 324.

The variable-length code decoder 324 decodes data of the I picture,which has been subjected to variable-length encoding and is suppliedfrom the buffer 321 through the selector 323, and supplies the data ofthe decoded I picture to a dequantization section 325. Thedequantization section 325 dequantizes the data of the I picture bymultiplying a dequantization coefficient equal to a predetermined valuefor each coefficient included in the data of the decoded I picture. Thedequantization section 325 supplies the data of the I picture obtainedthrough dequantization, that is, DCT (Discrete Cosine Transform)coefficients, to an inverse-DCT processing section 326.

The inverse-DCT processing section 326 generates uncompressed imagedata, so-called baseband image data, by applying inverse-DCT conversionto the DCT coefficients supplied form the dequantization section 325 andoutputs the baseband image data.

FIG. 37 is a block diagram depicting the structure of the resolutionconversion section 303. A low-pass filter 341 removes high-frequencycomponents (restricts the band) of the image from the baseband imagedata supplies from the I-picture selection and decoding section 302 andsupplies the baseband image data whose high-frequency components of theimage have been removed to a pixel-thinning section 342. For example,the low-pass filter 341 calculates the mean value of the pixel values offour pixels composed of two pixels in the vertical direction and twopixels in the horizontal direction and sets the calculated mean value tothe pixel values of the four pixels to remove the high-frequencycomponents of the image.

The pixel-thinning section 342 thins out pixels from the baseband imagedata whose high-frequency components of the image have been remove andoutputs the baseband image data whose pixels have been thinned out asthumbnail data. For example, the pixel-thinning section 342 classifiesthe pixels of the baseband image data into sets of four pixels composedof two pixels in the vertical direction and two pixels in the horizontaldirection and removes three pixels from each set of four pixels to thinout pixels from the baseband image data.

FIG. 38 is a block diagram depicting the structure of the JPEG encodingsection 304. A DCT processing section 361 DCT-converts the thumbnaildata supplied from the resolution conversion section 303 and suppliesthe DCT coefficients obtained through DCT conversion to a quantizationsection 362. The quantization section 362 quantizes the DCT coefficientsby dividing the DCT coefficients supplied from the DCT processingsection 361 by a predetermined quantization coefficient and supplies thequantized DCT coefficients to a variable-length encoding section 363.The variable-length encoding section 363 generates thumbnail datacompressed in accordance with the JPEG technique by encoding thequantized DCT coefficients into variable-length code and outputs thegenerated thumbnail data compressed in accordance with the JPEGtechnique.

FIG. 39 is a flowchart illustrating the process of generating thumbnaildata. In step S301, the I-picture selection and decoding section 302extracts (selects) an I picture from each GOP of moving image data,which is an MPEG2 program stream, read out by the drive 44 from the disk45. In step S302, the I-picture selection and decoding section 302decodes the extracted I picture.

In step S303, the resolution conversion section 303 performs conversionto decrease the resolution of the decoded I picture. In step S304, theJPEG encoding section 304 compresses the I picture with convertedresolution in accordance with the JPEG technique. In step S305, thefile-format conversion section 305 adjusts the file format of thethumbnail data acquired by compressing the I picture in accordance withthe JPEG technique, and the flow returns to step S301, where theabove-described processing is repeated.

The thumbnail data can also be generated as a stream of I pictures.

FIG. 40 is a block diagram depicting a structure of one embodiment of arecording and playback apparatus, according to the present invention,for reading out an MPEG2 program stream from the disk 45 on which theMPEG2 program stream is recorded and generating thumbnails, as a streamof I pictures, corresponding to the MPEG2 program stream to record it onthe disk 45. The same components as those shown in FIG. 35 are denotedby the same symbols, and thus a description thereof will be omitted.

An I-picture encoding section 381 compresses and encodes as an I picturethe thumbnail data with converted resolution, which is baseband imagedata and supplied from the resolution conversion section 303. TheI-picture encoding section 381 supplies the thumbnail data compressedand encoded as an I picture to the file-format conversion section 305.

FIG. 41 is a block diagram depicting the structure of the I-pictureencoding section 381. A buffer 401 temporarily stores the thumbnail datawith converted resolution, which is baseband image data and suppliedfrom the resolution conversion section 303. The buffer 401 supplies thestored thumbnail data to a visual-parameter detection section 402 and aDCT processing section 403.

The visual-parameter detection section 402 detects a visual parameterindicating a feature of the image of the thumbnail data stored in thebuffer 401 and supplies the detected visual parameter to a controlsection 404. The visual parameter can be, for example, an activityindicating a change of pixel value in the spatial direction, asspecified in MPEG2 TM(Test Model) 5.

The DCT processing section 403 DCT-converts the thumbnail data suppliedfrom the buffer 401 and supplies DCT coefficients obtained though DCTconversion to a quantization section 405.

The control section 404 determines a quantization value based on thevisual parameter supplied from the visual-parameter detection section402 and the amount of compressed thumbnail data stored in a buffer 407and supplies the quantization value to the quantization section 405. Forexample, as specified in MPEG2 TM5, the control section 404 determinesthe quantization value based on the visual parameter such that the morehigh-frequency components, the more coarsely quantization is performed,and the less high-frequency components, the more finely quantization isperformed. Furthermore, the control section 404 determines thequantization value based on the amount of compressed thumbnail datastored in the buffer 407 such that the data amount of thumbnail datacompressed and encoded as an I picture does not exceed a predeterminedupper limit.

The quantization section 405 quantizes the DCT coefficients by dividingthe DCT coefficients supplied from the DCT processing section 403 by thequantization value supplied from the control section 404 and suppliesthe quantized DCT coefficients to a variable-length encoding section406. The variable-length encoding section 406 generates thumbnail datacompressed and encoded as an I picture by encoding the quantized DCTcoefficients into variable-length code and supplies the generatedthumbnail data compressed and encoded as an I picture to the buffer 407.

The buffer 407 temporarily stores the thumbnail data compressed andencoded as an I picture. The buffer 407 outputs the stored thumbnaildata compressed and encoded as an I picture.

FIG. 42 is a flowchart illustrating the process of controlling theamount of code by the control section 404 for each I picture. In stepS321, the control section 404 assigns the amount of code to the picture.For example, in step S321, the control section 404 assigns the amount ofcode to the picture taking into consideration the upper limit of theamount of code for the picture and a predetermined margin rather thanthe target value of the amount of code for the picture. Morespecifically, in step S321, the control section 404 assigns to thepicture the amount of code of a value obtained by subtracting the marginfrom the upper limit of the amount of code.

This is done to guarantee the restrictions described in VBV forcontinuous playback of thumbnails, to be described later, because theamount of thumbnail data compressed and encoded as an I picture mayresults in exceeding the setting in step S321.

In step S322, the control section 404 assigns the amount of code tomacroblocks based on the amount of code assigned to the picture so as toallocate the amount of code assigned to the picture to each macroblock.

In step S323, the control section 404 determines the final quantizationvalue by using the visual parameter, and then processing ends.

As described above, the thumbnail data is compressed and encoded as an Ipicture so as to have an amount of data equal to or smaller than thepredetermined upper limit value. By doing so, even when thumbnail datais subjected to special playback, such as fast-forward or rewind, thethumbnail can be played back and displayed quickly without causing anunderflow in decoding.

This advantage will be described using a VBV (Video Buffering Verifier)model. The VBV is a model of a virtual decoder connected to the outputof an encoder, as specified in ISO13818-2 Annex C. It specifiesrestrictions to a bit stream based on constraints to the amount of datastored in a VBV buffer included in this model. Although the VBV normallyspecifies restrictions to the decoder, the current description assumesrestrictions to the encoding side.

FIG. 43 is a block diagram depicting the structure of a VBV model. Anencoder 421 outputs code corresponding to the picture to a VBV buffer422. The VBV buffer 422 temporarily stores the code supplied from theencoder 421 and outputs the stored code.

Here, it is assumed that the code is instantly transferred from theencoder 421 to the VBV buffer 422. It is also assumed that no code isoutput from the VBV buffer 422 if no code is stored in the VBV buffer422, and code is transferred from the VBV buffer 422 at the maximumtransfer rate if code is stored in the VBV buffer 422.

FIG. 44 is a diagram depicting the motion of a VBV model in a case whereno restriction is applied to the amount of thumbnail data. In FIG. 44,the vertical axis represents the amount of data of code stored in theVBV buffer 422, and the horizontal axis represents time.

In FIG. 44, time T indicates the frame period [second], which is equalto the inverse of the frame rate.

At time t=0 when encoding processing is started from a state where theVBV buffer 422 is empty, code with a data amount P0 is transferred fromthe encoder 421 to the VBV buffer 422. Accordingly, the data amount ofcode stored in the VBV buffer 422 becomes P0 at time t=0. Because thecode is output from the VBV buffer 422 at the maximum transfer rate, thedata amount of code stored in the VBV buffer 422 decreases over time,and the data amount of code stored in the VBV buffer 422 becomes B1 attime t=T. At the same time, since code with a data amount P1 smallerthan P0 is transferred from the encoder 421 to the VBV buffer 422 attime t=T, the data amount of code stored in the VBV buffer 422 instantlyincreases up to B1+P1.

Similarly, the data amount of code stored in the VBV buffer 422 becomesB2 at time t=2T. At the same time, since code with a data amount P2substantially the same as P1 is transferred from the encoder 421 to theVBV buffer 422 at time t=T, the data amount of code stored in the VBVbuffer 422 instantly increases up to B2+P2.

At time t=Tx, the data amount of code stored in the VBV buffer 422becomes zero. When the data amount of code stored in the VBV buffer 422becomes zero, the VBV buffer 422 outputs no code.

Since code with a data amount P3 smaller than P0 but larger than P1 istransferred from the encoder 421 to the VBV buffer 422 at time t=3T, thedata amount of code stored in the VBV buffer 422 instantly increases upto P3. At time t=4T, the data amount of code stored in the VBV buffer422 becomes zero, and at the same time, since code with a data amount P4substantially the same as P1 is transferred from the encoder 421 to theVBV buffer 422 at time t=T, the data amount of code stored in the VBVbuffer 422 instantly increases up to P4.

As described above, if the amount of code for one picture output fromthe encoder 421 changes, an underflow or an overflow may occur.

For example, code with a data amount P4 substantially the same as P1 istransferred from the encoder 421 to the VBV buffer 422 at time t=T andtime t=2T, an overflow occurs.

On the encoding side, an overflow is not allowed to occur because anoverflow causes the code to drop, whereas an underflow is allowed tooccur because writing can be awaited if an underflow occurs.

For this reason, an overflow needs to be prevented from occurring bymonitoring the amount of data encoded in encoding andfeedback-controlling the encoding processing by the use of a parameterfor changing the amount of code.

Here, a case is assumed where code with a data amount that can be outputat the maximum transfer rate is always transferred from the encoder 421to the VBV buffer 422 during a frame period T. This corresponds to, forexample, a case where, in step S321, the control section 404 assigns theamount of code to the picture by setting the amount of data that can beoutput at the maximum transfer rate during the frame period T as theupper limit of the amount of code for the picture.

Maximum permissible amount of code fb [bit/frame], which is the amountof data allowed for a thumbnail per frame (picture), can be calculatedas Rmax/frame_rate, where Rmax [bit/sec] is the maximum transfer rate ofcode output from the VBV buffer 422 and frame_rate [number offrames/sec] is the frame rate. It is noted that frame_rate differsdepending on the standards of television broadcasting, such as the NTSC(National Television System Committee) standards and the PAL (PhaseAlternating (by) Line) standards.

FIG. 45 is a diagram depicting the motion of a VBV model in a case wherethe data amount of each thumbnail is equal to the maximum permissibleamount of code fb calculated as described above. In FIG. 45, thevertical axis represents the amount of data of code stored in the VBVbuffer 422, and the horizontal axis represents time.

In FIG. 45, time T indicates the frame period [second], which is equalto the inverse of the frame rate.

At time t=0 when encoding processing is started from a state where theVBV buffer 422 is empty, code with a data amount P0 is transferred fromthe encoder 421 to the VBV buffer 422. Accordingly, the data amount ofcode stored in the VBV buffer 422 becomes P0 (=maximum permissibleamount of code fb) at time t=0. Because the code is output from the VBVbuffer 422 at the maximum transfer rate, the data amount of code storedin the VBV buffer 422 decreases over time, and the data amount of codestored in the VBV buffer becomes zero at time t=T. At the same time,since code with a data amount P1 equal to P0 is transferred from theencoder 421 to the VBV buffer 422 at time t=T, the data amount of codestored in the VBV buffer 422 instantly increases up to P1.

Because the code is output from the VBV buffer 422 at the maximumtransfer rate, the data amount of code stored in the VBV buffer 422decreases over time, and the data amount of code stored in the VBVbuffer 422 becomes zero at time t=2T. At the same time, since code witha data amount P2 equal to P0 is transferred from the encoder 421 to theVBV buffer 422 at time t=2T, the data amount of code stored in the VBVbuffer 422 instantly increases up to P2.

Similarly, the data amount of code stored in the VBV buffer 422 becomeszero during time t=3T to time t=nT. At the same time, since code with adata amount equal to P0 is transferred from the encoder 421 to the VBVbuffer 422, the data amount of code stored in the VBV buffer 422instantly increases up to the amount equal to P0.

In short, at a time when transfer from the encoder 421 to the VBV buffer422 occurs, the data amount of code stored in the VBV buffer 422 becomesmaximum, which is equal to the maximum permissible amount of code fb.

In the recording and playback apparatus whose structure is shown in FIG.40, the amount of thumbnail data is restricted to be equal to or belowthe maximum permissible amount of code fb, and therefore, an overflowoccurs less easily than with the structure shown in FIG. 45. In otherwords, FIG. 45 shows a state where the occupancy of the buffer ishighest.

A VBV for performing variable-rate playback of thumbnails will bedescribed in connection with the playback side. In variable rateplayback, arbitrary thumbnails (pictures) are played back in arbitraryorder. Therefore, if the amount of data of each thumbnail changes,playback of thumbnails with a large amount of data may be requestedsequentially. In this case, the occupancy of the VBV buffer may decreaseto such a level that the VBV buffer becomes empty, thus causing anunderflow. Consequently, the playback of thumbnails delays, causingdisplay of thumbnails not be switched as instructed by the user.

By limiting the data amount of all thumbnails equal to or below themaximum permissible amount of code fb, the occupancy of the VBV bufferdoes not fall below a certain level, whichever thumbnails are playedback in whatever order, thus preventing an underflow. Consequently, itis ensured that display of thumbnails is switched as instructed by theuser.

FIG. 46 is a block diagram depicting another structure of one embodimentof a recording and playback apparatus, according to the presentinvention, for reading out an MPEG2 system stream from the disk 45 onwhich the MPEG2 system stream is recorded and generating thumbnails, asa stream of I pictures, corresponding to the MPEG2 system stream torecord it on the disk 45. The same components as those shown in FIG. 40are denoted by the same symbols, and thus a description thereof will beomitted.

Under the control of the microcomputer 31, an I-picture selection anddecoding section 451 selects an I (intra) picture from among the MPEG2video elementary stream separated by the demultiplexer 202 and decodesthe selected I picture to DCT coefficients. The I-picture selection anddecoding section 451 supplies the decoded DCT coefficients to afrequency-characteristic conversion section 452.

Under the control of the microcomputer 31, the frequency-characteristicconversion section 452 converts the frequency characteristic of thedecoded DCT coefficients. For example, from among the DCT coefficients,the frequency-characteristic conversion section 452 converts thefrequency characteristic of the DCT coefficients by removing orattenuating components corresponding to high frequencies of the image.

The frequency-characteristic conversion section 452 supplies the DCTcoefficients whose frequency characteristic has been converted to anI-picture encoding section 453. An I-picture encoding section 453compresses and encodes as an I picture the DCT coefficients withconverted frequency characteristic supplied from thefrequency-characteristic conversion section 452. The I-picture encodingsection 453 supplies thumbnail data obtained by compressing and encodingthe DCT coefficients as an I picture to a file-format conversion section305.

FIG. 47 is a block diagram depicting the structure of the I-pictureselection and decoding section 451. A buffer 471 temporarily stores themoving image data in accordance with the MPEG2 technique, which is anMPEG2 video elementary stream, supplied from the demultiplexer 202. AnI-picture determination section 472 determines whether each of thepictures constituting the moving image data in accordance with the MPEG2technique stored in the buffer 471 is an I picture by, for example,referring to the picture coding type in the picture header.

A selector 473 supplies data of the picture stored in the buffer 471 toa variable-length code decoder 474 or controls the supply of the data ofpicture stored in the buffer 471 to the variable-length code decoder 474based on a signal indicating whether the picture is an I picturesupplied from the I-picture determination section 472. Morespecifically, if a signal indicating that the picture is an I picture issupplied from the I-picture determination section 472, the selector 473supplies data of the picture, which is an I picture stored in the buffer471, to the variable-length code decoder 474. If a signal indicatingthat the picture is not an I picture is supplied from the I-picturedetermination section 472, the selector 473 controls the supply of dataof the picture, which is a B picture or a P picture stored in the buffer471, to the variable-length code decoder 474.

The variable-length code decoder 474 decodes data of the I picture,which has been subjected to variable-length encoding and is suppliedfrom the buffer 471 through the selector 473, and supplies the data ofthe decoded I picture to a dequantization section 475. Thedequantization section 475 dequantizes the data of the I picture bymultiplying a dequantization coefficient equal to a predetermined valuefor each coefficient included in the data of the decoded I picture. Thedequantization section 475 outputs the data of the I picture obtainedthrough dequantization, that is, DCT coefficients and a quantizationscale.

FIG. 48 is a block diagram depicting the structure of thefrequency-characteristic conversion section 452. A horizontal filter 491removes or attenuates high-frequency components for DCT coefficientsarranged in the horizontal direction from among the DCT coefficients.

As shown in FIG. 49, DCT coefficients are arranged two dimensionally,where the order in the horizontal direction (n) is 0 to 7 and the orderin the vertical direction (m) is 0 to 7. DCT coefficients with higherorders in the horizontal direction (n) correspond to higher frequencycomponents of the image. DCT coefficients with higher orders in thevertical direction (m) correspond to higher frequency components of theimage.

FIG. 50 is a diagram depicting a transfer function H(n) of thehorizontal filter 491 and a transfer function V(m) of a vertical filter492. For example, the horizontal filter 491 sets DCT coefficients withhigher orders (n) in the horizontal direction as 0 or decreases theirvalues based on the transfer function H(n) shown in FIG. 50 to remove orattenuate high-frequency components for the DCT coefficients arranged inthe horizontal direction from among the DCT coefficients.

For the DCT coefficients arranged in the horizontal direction, thehorizontal filter 491 supplies DCT coefficients whose high-frequencycomponents have been removed or attenuated to the vertical filter 492.

The vertical filter 492 remove or attenuates high-frequency componentsfor DCT coefficients arranged in the vertical direction from among theDCT coefficients supplied from the horizontal filter 491 and outputsthem. For example, the vertical filter 492 sets DCT coefficients withhigher orders (m) in the vertical direction, supplied from thehorizontal filter 491, as 0 or decreases their values based on thetransfer function V(m) shown in FIG. 50 to remove or attenuatehigh-frequency components for the DCT coefficients arranged in thevertical direction from among the DCT coefficients.

More specifically, DCT coefficients d′(n,m) calculated asd(n,m)×H(n)×V(m) are output from the frequency-characteristic conversionsection 452 whose structure is shown in FIG. 48, where d(n,m) means DCTcoefficients input to the frequency-characteristic conversion section452. Because DCT coefficients are values in the frequency domain,filtering processing can be performed by multiplying the above-describedtransfer functions.

In other words, restrictions in the frequency domain of the image can beapplied to a block composed of eight pixels in the vertical directionand eight pixels in the horizontal direction by multiplying 8×8 DCTcoefficients by the above-described transfer function H(n) and transferfunction V(m).

The quantization scale output from the I-picture selection and decodingsection 451 passes through the frequency-characteristic conversionsection 452 as-is and is input to the I-picture encoding section 453.

FIG. 51 is a block diagram depicting the structure of the I-pictureencoding section 453. A control section 501 determines a quantizationvalue based on the quantization scale supplied from the I-pictureselection and the decoding section 451 through thefrequency-characteristic conversion section 452 and the amount ofcompressed thumbnail data stored in a buffer 504 and supplies thequantization value to a quantization section 502. For example, thecontrol section 501 determines a quantization value based on the amountof compressed thumbnail data stored in the buffer 504 such that the dataamount of thumbnail data compressed and encoded as an I picture does notexceed the predetermined upper limit.

The quantization section 502 quantizes DCT coefficients by dividing theDCT coefficients supplied from the frequency-characteristic conversionsection 452 by the quantization value supplied from the control section501 and supplies the quantized DCT coefficients to a variable-lengthencoding section 503. Since high-frequency components of the DCTcoefficients input to the I-picture encoding section 453 have beenremoved or attenuated, the DCT coefficients are small values comparedwith DCT coefficients before they are subject to frequencycharacteristic conversion. Therefore, a larger number of DCTcoefficients with a value of 0 exist for re-quantization by thequantization section 502, and the truncation order becomes small,accordingly.

The variable-length encoding section 503 generates thumbnail datacompressed and encoded as an I picture by encoding the quantized DCTcoefficients into variable-length code and supplies the generatedthumbnail data compressed and encoded as ad I picture to the buffer 504.

The buffer 504 temporarily stores the thumbnail data compressed andencoded as an I picture. The buffer 504 outputs the stored thumbnaildata compressed and encoded as an I picture.

Since high-frequency components of the DCT coefficients input to theI-picture encoding section 453 have been removed or attenuated, theI-picture encoding section 453 can output thumbnail data with a smalleramount of data.

FIG. 52 is a flowchart illustrating another process of generatingthumbnail data. In step S361, the I-picture selection and decodingsection 451 extracts (selects) an I picture from each GOP of movingimage data, which is an MPEG2 program stream, read out by the drive 44from the disk 45. In step S362, the I-picture selection and decodingsection 451 decodes the extracted I picture to DCT coefficients.

In step S363, the frequency-characteristic conversion section 452removes high-order components of the decoded DCT coefficients to convertthe frequency characteristic of the DCT coefficients. In step S364, theI-picture encoding section 453 compresses and encodes the DCTcoefficients whose frequency characteristic has been converted as an Ipicture. In step S365, the file-format conversion section 305 adjuststhe file format of the thumbnail data compressed and encoded as an Ipicture, and the flow returns to step S361, where the above-describedprocessing is repeated.

FIG. 53 is a flowchart illustrating the process of controlling theamount of code by the control section 501 for each I picture. In stepS381, the control section 501 assigns the amount of code to the picture.For example, in step S381, the control section 501 assigns the amount ofcode to the picture taking into consideration the upper limit of theamount of code for the picture and a predetermined margin rather thanthe target value of the amount of code for the picture. Morespecifically, in step S381, the control section 501 assigns to thepicture the amount of code of a value obtained by subtracting the marginfrom the upper limit of the amount of code.

This is done to guarantee the restrictions described in VBV forcontinuous playback of thumbnails, as described above, because theamount of thumbnail data compressed and encoded as an I picture mayresults in exceeding the setting in step S321.

In step S382, the control section 501 determines the final quantizationvalue based on the amount of code assigned to the picture and assignsthe amount of code to macroblocks, and then the processing ends.

Next, a recording and playback apparatus for reading an MPEG2 systemstream from the disk 45 recording the MPEG2 system stream, readingthumbnail data from the disk 45 on which thumbnail data generatedcorresponding to the MPEG2 system stream is recorded, and playing backthumbnails will be described.

FIG. 54 is a block diagram depicting another structure of one embodimentof a playback block in a recording and playback apparatus, according tothe present invention, for playing back and displaying thumbnails basedon the thumbnail data 81 compressed and encoded in accordance with theJPEG technique. The same components as those shown in FIG. 20 aredenoted by the same symbols, and thus a description thereof will beomitted.

Under the control of the microcomputer 31, a file-format conversionsection 521 reads out the thumbnail data 81 which has been read out bythe drive 44 from the disk 45 into the buffer memory 43, where the fileformat of the thumbnail data 81 is a PLF format, a file format referredto from the file 101 in PLF format, a static image package format, alocation-related data file format referred from the track managementfile 131, or a file format referred to from the location-related datafile 141.

The file-format conversion section 521 converts the file format of theread-out thumbnail data 81 and supplies the thumbnail data 81 withconverted file format to a JPEG decoding section 522. For example, thefile-format conversion section 521 converts the file format of thethumbnail data 81 by extracting the thumbnail data 81 compressed andencoded in accordance with the JPEG technique from the thumbnail data 81in a PLF format, a file format referred to from the file 101 in PLFformat, a static image package format, a location-related data fileformat referred to from the track management file 131, or a file formatreferred to from the location-related data file 141.

Under the control of the microcomputer 31, the JPEG decoding section 522decodes the thumbnail data 81 compressed and encoded in accordance withthe JPEG technique, supplied from the file-format conversion section521, and stores in the buffer memory 35 the thumbnail data 81, as abaseband image, obtained through decoding.

FIG. 55 is a block diagram depicting the structure of the JPEG decodingsection 522. A variable-length code decoder 541 decodes the thumbnaildata 81 subjected to variable-length encoding, supplied from thefile-format conversion section 521, and supplies the decoded thumbnaildata 81 to a dequantization section 542. The dequantization section 542dequantizes the thumbnail data 81 by multiplying a dequantizationcoefficient equal to a predetermined value for each coefficient includedin the decoded thumbnail data 81. The dequantization section 542supplies the thumbnail data 81 obtained through dequantization, that is,DCT coefficients, to an inverse-DCT processing section 543.

The inverse-DCT processing section 543 generates uncompressed imagedata, so-called baseband image data, by applying inverse-DCT conversionto the DCT coefficients supplied form the dequantization section 542 andoutputs the baseband image data.

FIG. 56 is a block diagram depicting another structure of one embodimentof a playback block in a recording and playback apparatus, according tothe present invention, for playing back and displaying thumbnails basedon the thumbnail data 81 compressed and encoded as a stream of Ipictures. The same components as those shown in FIG. 20 are denoted bythe same symbols, and thus a description thereof will be omitted.

Under the control of the microcomputer 31, a file-format conversionsection 561 reads out the thumbnail data 81 which has been read out bythe drive 44 from the disk 45 into the buffer memory 43, where the fileformat of the thumbnail data 81 is a PLF format, a file format referredto from the file 101 in PLF format, a static image package format, alocation-related data file format referred from the track managementfile 131, or a file format referred to from the location-related datafile 141.

The file-format conversion section 561 converts the file format of theread-out thumbnail data 81 and supplies the thumbnail data 81 withconverted file format to an I-picture decoding section 562. For example,the file-format conversion section 561 converts the file format of thethumbnail data 81 by extracting the thumbnail data 81 compressed andencoded as a stream of I pictures from the thumbnail data 81 in a PLFformat, a file format referred to from the file 101 in PLF format, astatic image package format, a location-related data file formatreferred to from the track management file 131, or a file formatreferred to from the location-related data file 141.

Under the control of the microcomputer 31, the I-picture decodingsection 562 decodes the thumbnail data 81 compressed and encoded as astream of I pictures, supplied from the file-format conversion section561, and stores in the buffer memory 35 the thumbnail data 81, as abaseband image, obtained through decoding.

FIG. 57 is a block diagram depicting the structure of the I-picturedecoding section 562. A variable-length code decoder 581 decodes data ofthe I picture, which has been subjected to variable-length encoding andis supplied from the file-format conversion section 561, and suppliesthe data of the decoded I picture to a dequantization section 582. Thedequantization section 582 dequantizes the data of the I picture bymultiplying a dequantization coefficient equal to a predetermined valuefor each coefficient included in the data of the decoded I picture. Thedequantization section 582 supplies the data of the I picture obtainedthrough dequantization, that is, DCT coefficients, to an inverse-DCTprocessing section 583.

The inverse-DCT processing section 583 generates uncompressed imagedata, so-called baseband image data, by applying inverse-DCT conversionto the DCT coefficients supplied form the dequantization section 582 andoutputs the baseband image data.

If the thumbnail data 81 is subjected to frequency characteristicconversion such that, for example, high-frequency components of theimage are removed and is compressed and encoded as a stream of Ipictures, pixels of thumbnails may be thinned out for display.

FIG. 58 is a block diagram depicting still another structure of oneembodiment of a playback block in a recording and playback apparatus,according to the present invention, for playing back and displayingthumbnails based on the thumbnail data 81 compressed and encoded as astream of I pictures. The same components as those shown in FIG. 56 aredenoted by the same symbols, and thus a description thereof will beomitted.

A pixel-thinning section 591 thins out pixels at predetermined positionsfrom among pixels of the thumbnail data 81, as baseband image data,supplied from an I-picture decoding section 562 and stores the thumbnaildata 81 whose pixels have been thinned out in the buffer memory 35. Forexample, the pixel-thinning section 591 classifies the pixels of thethumbnail data 81 into sets of four pixels composed of two pixels in thevertical direction and two pixels in the horizontal direction andremoves three pixels from each set of four pixels to thin out pixelsfrom the thumbnail data 81.

If the thumbnail data 81 is subjected to frequency characteristicconversion such that high-frequency components of the image are removedand is compressed and encoded as a stream of I pictures, the image sizeis not reduced. Thus, the pixel-thinning section 591 can performthinning to reduce the image size.

If there is no problem of aliasing in the thumbnail data 81 whosefrequency characteristic has been converted such that high-frequencycomponents of the image are removed, the pixel-thinning section 591 cansimply thin out pixels. If there is a problem of aliasing, thepixel-thinning section 591 should preferably restrict the band of imagedata through a low-pass filter before thinning out pixels.

If a picture according to a moving image is generated as describedabove, the picture according to the moving image can be recorded on adata recording medium. Furthermore, if one picture is extracted from aunit in which a moving image is encoded where the unit is composed of aconstant number of pictures; the amount of information of the extractedpicture is reduced; the picture whose amount of information has beenreduced is encoded by a predetermined encoding scheme; the encodedpicture is associated with the unit from which the picture has beenextracted; and recording of the picture associated with the unit onto adata recording medium on which the moving image is to be recorded iscontrolled, then the picture associated with the unit can be quicklyplayed back when the moving image is to be played back. As a result, theuser can view the content at a desired point in time for playback of themoving image.

In addition, if the picture recorded on a data recording medium, i.e.,the picture according to the moving image, is read out, the pictureaccording to the moving image can be played back. Furthermore, ifreading a picture from a data recording medium recording a moving imageand the picture is controlled; wherein the picture is extracted from aunit in which the moving image is encoded, the unit including a constantnumber of pictures, the amount of information of the picture is reduced,the picture is encoded by a predetermined encoding scheme, the pictureis associated with each unit, and the reading is based on an instructionfrom a user and a relationship with the unit of the moving image; theread-out picture is decoded; and display of the decoded picture iscontrolled, then the picture associated with the unit can be played backquickly. As a result, the user can view the content at a desired pointin time for playback of the moving image.

A technique for compressing and encoding thumbnails is not limited tothe JPEG technique or encoding as I pictures. Encoding schemes that cancontrol the data amount of each thumbnail, such as the JPEG2000 and themotion JPEG, are acceptable. In addition, although the encoding schemeof a moving image is MPEG2 in the above description, the encoding schemeof a moving image is not limited to MPEG2. Other encoding schemesincluding MPEG4 and MPEG7 are also acceptable.

The sequence of processing described above can be implemented using notonly hardware but also software. If the series of processing is to becarried out with software, a program constituting the software isinstalled from a recording medium into a computer built in dedicatedhardware or into, for example, a general-purpose personal computer whichcan carry out various types of functions by installing various types ofprograms.

As shown in FIG. 3, FIG. 20, FIG. 35, FIG. 40, FIG. 46, FIG. 54, FIG.56, or FIG. 58, this recording medium containing the program may be thedisk 45 which is a package medium including a magnetic disk (including aflexible disk), an optical disk (including CD-ROM (Compact Disc-ReadOnly Memory) and DVD (Digital Versatile Disc), a magneto-optical disk(including MD (Mini-Disc)™), or a semiconductor memory if such a programis supplied separately from a user's computer. The recording medium maybe a ROM (not shown) built into a microcomputer or a hard disk (notshown) in a computer if the program on the recording medium is suppliedpreinstalled on the computer.

The program that carries out the above-described sequence of processingmay be installed in a computer through an interface, such as a router ora modem, as required and through a wired or wireless communicationmedium, such as a local area network, the Internet, or digital satellitebroadcasting.

In the present invention, the steps of the program recorded on therecording medium may or may not be followed time-sequentially in orderof described steps. Furthermore, the steps may be followed in parallelor independently from one another.

1. A hardware recording apparatus comprising: extraction means forextracting an image from a unit in which a constant number of movingimages are included; reduction means for reducing the amount ofinformation of the extracted image; encoding means for encoding theextracted image with reduced amount of information by a predeterminedencoding scheme; association means for associating the encoded imagewith the unit from which the image is extracted by the extraction means;and recording control means for controlling recording of the encodedimage associated with the unit and controlling recording of movingimages onto a data recording medium, wherein the recording control meansis configured to: control storing of moving images data in a firstbuffer and storing of encoded images data in a second buffer; determinean amount of moving images data stored in the first buffer; controlrecording of moving images data in a first contiguous area of the datarecording medium when the amount of moving images data stored in thefirst buffer is no less than a first predetermined threshold, whereinthe recording of the moving images data in the first contiguous area isin parallel to the storing of encoded images data in the second buffer;after the moving images data recorded in the first contiguous areareaches a predetermined limit associated with the first contiguous area,stop the recording of the moving images data in the first contiguousarea and determine an amount of encoded images data stored in the secondbuffer; and control recording of encoded images data in a secondcontiguous area of the data recording medium when the amount of encodedimages data stored in the second buffer is no less than a secondpredetermined threshold, wherein the recording of the encoded imagesdata in the second contiguous area is in parallel to the storing ofmoving images data in the first buffer.
 2. The recording apparatusaccording to claim 1, wherein the association means is a trackassociated with a track of the moving image and associates the encodedimage with the unit by arranging the encoded image in a track in apredetermined file format.
 3. The recording apparatus according to claim1, wherein the association means associates the encoded image with theunit by associating a range of time for playback of the unit of themoving image with the encoded image.
 4. The recording apparatusaccording to claim 1, wherein the encoding means encodes the image by acompression and encoding scheme for a static image.
 5. The recordingapparatus according to claim 1, wherein the encoding means encodes theimage by a compression and encoding scheme for a moving image such thatdecoding is possible only with the image.
 6. The recording apparatusaccording to claim 1, wherein the reduction means reduces the amount ofinformation of the image by thinning out pixels of the image.
 7. Therecording apparatus according to claim 1, wherein the reduction meansreduces the amount of information of the image by removing ahigh-frequency component of the image.
 8. A recording method comprising:an extraction step of extracting an image from a unit in which aconstant number of moving images are included; a reduction step ofreducing the amount of information of the extracted image; an encodingstep of encoding the extracted image with reduced amount of informationby a predetermined encoding scheme; an association step of associatingthe encoded image with the unit from which the image is extracted in theextraction step; and a recording control step of controlling recordingof the encoded image associated with the unit and controlling recordingof moving images onto a data recording medium, wherein the recordingcontrol step comprises: controlling storing of moving images data in afirst buffer and storing of encoded images data in a second bufferdetermining an amount of moving images data stored in the first buffer;controlling recording of moving images data in a first contiguous areaof the data recording medium when the amount of moving images datastored in the first buffer is no less than a first predeterminedthreshold, wherein the recording of the moving images data in the firstcontiguous area is in parallel to the storing of encoded images data inthe second buffer; after the moving images data recorded in the firstcontiguous area reaches a predetermined limit associated with the firstcontiguous area, stopping the recording of the moving images data in thefirst contiguous area and determining an amount of encoded images datastored in the second buffer; and controlling recording of the encodedimages data in a second contiguous area of the data recording mediumwhen the amount of encoded images data stored in the second buffer is noless than a second predetermined threshold, wherein the recording of theencoded images data in the second contiguous area is in parallel to thestoring of moving images data in the first buffer.
 9. A non-transitoryrecording medium storing a program causing a computer to performrecording processing comprising: an extraction step of extracting animage from a unit in which a constant number of moving images areincluded; a reduction step of reducing the amount of information of theextracted image; an encoding step of encoding the extracted image withreduced amount of information by a predetermined encoding scheme; anassociation step of associating the encoded image with the unit fromwhich the image is extracted in the extraction step; and a recordingcontrol step of controlling recording of the encoded image associatedwith the unit and controlling recording of moving images onto a datarecording medium, wherein the recording control step comprises:controlling storing of moving images data in a first buffer and storingof encoded images data in a second buffer determining an amount ofmoving images data stored in the first buffer; controlling recording ofmoving images data in a first contiguous area of the data recordingmedium when the amount of moving images data stored in the first bufferis no less than a first predetermined threshold, wherein the recordingof the moving images data in the first contiguous area is in parallel tothe storing of encoded images data in the second buffer; after themoving images data recorded in the first contiguous area reaches apredetermined limit associated with the first contiguous area, stoppingthe recording of the moving images data in the first contiguous area anddetermining an amount of encoded images data stored in the secondbuffer; and controlling recording of the encoded images data in a secondcontiguous area of the data recording medium when the amount of encodedimages data stored in the second buffer is no less than a secondpredetermined threshold, wherein the recording of the encoded imagesdata in the second contiguous area is in parallel to the storing ofmoving images data in the first buffer.
 10. A hardware playbackapparatus comprising: reading control means for controlling reading animage from a data recording medium recording moving images and theimage, the image being extracted from a unit in which a constant numberof moving images are included, the amount of information of the imagebeing reduced, the image being encoded by a predetermined encodingscheme, the image being associated with the unit, moving images databeing stored in a first buffer and encoded images data being stored in asecond buffer, the moving images data being recorded onto the datarecording medium when an amount of moving images data stored in thefirst buffer is determined to be no less than a first predeterminedthreshold such that the moving images data are recorded in a firstcontiguous area of the data recording medium in parallel to the storingof encoded images data in the second buffer, the encoded images databeing recorded in a second contiguous area of the data recording mediumin parallel to the storing of moving images data in the first buffer andafter the moving images data recorded in the first contiguous areareaches a predetermined limit associated with the first contiguous areaand the recording of the moving images data in the first contiguous areastops, and when an amount of encoded images data stored in the secondbuffer is determined to be no less than a second predeterminedthreshold, and the reading being based on an instruction from a user anda relationship with the unit; decoding means for decoding the image; anddisplay control means for controlling display of the decoded image. 11.The playback apparatus according to claim 10, wherein the readingcontrol means controls reading the image from the data recording mediumso as to read only the image if the user directs a fast-forwardoperation or a rewind operation.
 12. The playback apparatus according toclaim 10, wherein the decoding means decodes the image encoded by acompression and encoding scheme for a static image.
 13. The playbackapparatus according to claim 10, wherein the decoding means decodes theimage encoded by a compression and encoding scheme for the moving imagesuch that decoding is possible only with the image.
 14. A playbackmethod comprising: a reading control step of controlling reading animage from a data recording medium recording moving images and theimage, the image being extracted from a unit in which a constant numberof moving images are included, the amount of information of the imagebeing reduced, the image being encoded by a predetermined encodingscheme, the image being associated with the unit, moving images databeing stored in a first buffer and encoded images data being stored in asecond buffer, the moving images data being recorded onto the datarecording medium when an amount of moving images data stored in thefirst buffer is determined to be no less than a first predeterminedthreshold such that the moving images data are recorded in a firstcontiguous area of the data recording medium in parallel to the storingof encoded images data in the second buffer, the encoded images databeing recorded in a second contiguous area of the data recording mediumin parallel to the storing of moving images data in the first buffer andafter the moving images data recorded in the first contiguous areareaches a predetermined limit associated with the first contiguous areaand the recording of the moving images data in the first contiguous areastops, and when an amount of encoded images data stored in the secondbuffer is determined to be no less than a second predeterminedthreshold, and the reading being based on an instruction from a user anda relationship with the unit; a decoding step of decoding the image; anda display control step of controlling display of the decoded image. 15.A non-transitory recording medium storing a program causing a computerto perform playback processing comprising: a reading control step ofcontrolling reading an image from a data recording medium recordingmoving images and the image, the image being extracted from a unit inwhich a constant number of moving images are included, the amount ofinformation of the image being reduced, the image being encoded by apredetermined encoding scheme, the image being associated with the unit,moving images data being stored in a first buffer and encoded imagesdata being stored in a second buffer, the moving images data beingrecorded onto the data recording medium when an amount of moving imagesdata stored in the first buffer is determined to be no less than a firstpredetermined threshold such that the moving images data are recorded ina first contiguous area of the data recording medium in parallel to thestoring of encoded images data in the second buffer, the encoded imagesdata being recorded in a second contiguous area of the data recordingmedium in parallel to the storing of moving images data in the firstbuffer and after the moving images data recorded in the first contiguousarea reaches a predetermined limit associated with the first contiguousarea and the recording of the moving images data in the first contiguousarea stops, and when an amount of encoded images data stored in thesecond buffer is determined to be no less than a second predeterminedthreshold, and the reading being based on an instruction from a user anda relationship with the unit; a decoding step of decoding the image; anda display control step of controlling display of the decoded image.